o 


TITLE:  Equipment,  Personnel,  Facilities  and  Supplies  to  Conduct 

Studies  on  the  Research  Project  Entitled  "  Look  and  Feel: 
Haptic  Interaction  for  Biomedicine" 


PRINCIPAL  INVESTIGATOR:  J.  Kenneth  Salisbury 


CONTRACTING  ORGANIZATION:  Massachusetts  Institute  of  Technology 

Cambridge,  Massachusetts  02139 


REPORT  DATE:  October  1995 


TYPE  OF  REPORT:  Annual 


PREPARED  FOR:  U.S.  Army  Medical  Research  and  Materiel  Command 
Fort  Detrick,  Maryland  21702-5012 


DISTRIBUTION  STATEMENT: 


Approved  for  public  release;  • 

distribution  unlimited 


The  views,  opinions  and/or  findings  contained  in  this  report  are 

those  of  the  author (s)  and  should  not  be  construed  as  an  official  • 

Department  of  the  Army  position,  policy  or  decision  unless  so 

designated  by  other  documentation. 


95-02466 

■11111111 


ro 


» 


October  1995 


Annual  1  Sep  94  -  31  Aug  95 


Equipment,  Personnel,  Facilities  and  Supplies  to  Conduct 
Studies  on  the  Research  Project  Entitled:  "Look  and  Feel: 
Haptic  Interaction  for  Biomedicine" 


DAMD17-94-C-4123 


J.  Kenneth  Salisbury 

Massachusetts  Institute  of  Technology 
Cambridge,  Massachusetts  02139 


U.S.  Army  Medical  Research  and  Materiel  Command 
Fort  Detrick,  Maryland  21702-5012 


Approved  for  public  release;  distribution  unlimited 


This  project  develops  and  demonstrates  key  technologies  for  advanced 
virtual  interaction  for  surgical  simulation  and  teleoperation  systems 
used  to  perform  remote  surgical  procedures.  Our  focus  at  MIT  during  the 
past  year  has  been  to  develop  the  first  level  of  interfaces  and  algo¬ 
rithms  needed  to  accomplish  the  broad  goals  of  performing  virtual  and 
remote  surgical  procedures.  During  the  past  year  we  have  implemented  a 
computational  environment  including  control  software,  graphics  software 
and  computer  hardware,  to  support  the  development  of  haptic  rendering 
algorithms.  We  have  demonstrated  tweezer  interaction  with  flat  compli¬ 
ant  surfaces  to  simulate  grasping  of  tissue  and  developed  a  real-time, 
reduced  order,  finite-element  simulation  of  compliant  materials.  The 
resulting  systems  has  been  used  successfully  to  achieve  the  year-one 
goal  of  demonstrating  multi-finger  manipulation  of  virtual  compliant 
material.  In  addition,  we  have  performed  preliminary  investigation  and 
design  of  a  remote  palpation  and  manipulation  device  which  will  be  used 
in  the  coming  years  to  perform  real  surgical  procedures. 


haptics,  human  computer  interface,  surgical 
simulation 


Unclassified 


Unclassified 


Unclassified 


Unlimited 


1 


GENERAL  jNb,l  RUCTIONS  :;OR  COMPLETING  St-  298 

The  Report  Document -ition  Page  (RDP)  h  used  -n  amount  mg  and  cataloging  reports  It  is  important 
that  this  information  be  consistent  wn  the*  rest  of  the  report,  particularly  the  cover  and  title  page. 
Instruct!  ns  for  filling  in  each  block  of  the  form  tollow  ‘t  is  important  to  stay  within  the  lines  to  meet 
optical  scanning  requirements 


Block  I  Agency  Use  Only  ii  nave  frank} 

Block  2.  Report  Date.  Full  publication  date 
including  day,  month,  sod  year,  rt  available  mg  1 
Jan  38)  Must  tile  at  h  ,>st  Li  it?  war 

Block  3  Type  of  R epc>T  arid  fates  Cqvereg 
Stale  whetitei  report  a,  .nten.n,  f-nal,  etc  if 
applicable,  ppwv  i"cl\js,ve  report  dates  •{:  o  o 
jun  8/  it!  tun  38). 

Block  4.  '  ;tje  and  Suotiue  A  title  token  bom 

the  part,  ot  the  ’ noon  r l ;  nu-ujfU  •>  tin-  in.  . 
meaningful  and  i.ornp  eie  information  When  a 
repeat  'S  prvp.u eb  n*  mpr  ■:  '.nun  ^  .m  vr.t. 
repeat  the  p.ima'y  usm  mm  :»:!'»*•  io :  uno 
inch.icJe  sutvrle  nv  <}>►>. .  ; 
classified  dor cir'Mvts  enter  **'c  :  tie  t-c.it  on 
in  parentheses 

Blocks  f...u:.:-j  ,:i  »v  \ .  Vc  I:  U-.i 

and  gram  numbr;-.,  m.iy  i-  .  mde  nam.  am 
elemeni  nuMbe*  ip  pn.vei'  'm •<••  •: j'--:  ’•,)  ,.;s!t 


numiier(s),  o w.-vl  u. 

.a  •  I.,. > ripe* r( s; .  Jse  t-->e 

foliowrni;  iaiar-D 

C  ■"  *  h‘ !  r ,  r  • 

fk 

r,  .  r 

T4  !,:<•. 

PE 

WLi  War f  unu 

i':0O'c;t  l . 

•  i:‘  ‘i  n : 

Block  12a.  Dirtribution/Avai  I  ability  Statement. 
Denotes  public  availability  or  limitations.  Cite  any 
availability  to  the  public  Enter  additional 
limitations  or  special  markings  in  all  capitals  (e  g. 
Nt>  OMN.  Iff-'.,  I  TAR) 

OOD  See  UoDD  5230.24.  "Distribution 
Statements  on  Technical 
Documents  " 

0  O  t.  S  e  c  (Ki  t  h  o  r :  t 1  es 

NASA  See  Handbook  NHB  2200.2. 

N  US  •  Leave  blank. 

Slock  1?b.  Disfnbiition  Code 

DOD  leave  blank.  , 

OOF  -  Enter  DOF.  distribution  categories 

•torn  the  Standard  Distribution  for  •.ft 
linc'asss bed  Scientific  and  Technical 
be  ports. 

d ASA  Leave  olank. 

N  ilS  Leave  blank. 

B'oi  k  ■  i  "ns-tgef  include  a  brief  (Maximum 
/JO  wonts)  tual  Summary  the  most 
Signr. leant  information  contained  m  the  report. 


Block 

1  i 

■  *  ) 

K’  i  - ; 

.  '  ■> :  lH1  ' 

.  O  ' 

respo 

ih-il 

•I.*5  *yv 

;  -,i  .  ■  ir- 

• »  •  ■  t’i  i 

i.-. 

ihe  :  C 

'  ’>  ii'  -  • ! 

.  r 

:  f  i 

t  1  A, 

*  1  •  • . 

repo-' 

,  ;  | 

rM.l 

•’cr  <: 

};  (  • 

(h;s 

If.i 

».  ho  n, 

) 

Block 

7  r 

1  'r* 

: 

Or;;,: 

•»;# 

\.Ws  t  (  .}  .it'd 

Audi . 

-  *.* 

•>  i 

.■><;*!  * 

■'  A  r 

/U»‘  1«H 

Block 

8.  t 

? ;  .  ( • ' 

■:nt 

fp/  'idfi; 

Numb 

}&  E 

'  ■  n 

1? 

•  i  *  4  ( )  *  ’ 

r 

nci  if  '.'O' u  i 

numrj 

ni’ls) 

as 

:d  t 

•V 

:  1 0  1 

per  ft) 

.*  :'T*! n 

a  i 

1 ! » 5  ■  \ 

■p 

f  1 

Blocks  S 

’££ 

irhor 

t:  1-5 

r  Mon 

!  (>•  inn  Ac 

jpn.i.y  Ngnie(s) 

and  / 

1  igr; 

ii>  ■?) 

Sei 

(  exu! 

<w  ic'.i  ( ( if  y 

Block 

10. 

s 

pone. 

t>r  • 

•M^7l-> 

r 

f. ;  ^  y 

Reooi 

1  N  i 

i  p  ■ 

\>f>r 

i\1 

*e;»t  -■ ;  v 

»1  j 

Block  'I  1L:U| •  •iMaiy  :  r-ti  • 

P'.jb!- •' 

r.i  a  t  d  t  f?  r  H  8  !  ’  l  VV  *  *  l  (->  .  ■  it  '  ;  i -  -Hg  ,,  sr;  .  I  T  »  /  t  *  {  ‘  < 


H‘-o:  :<  14  Solves i  l  erpis  Keywords  or  phrases 
i subjects  in  the  report 

H'ock  IS  Hum  i-'gcjcs  Enter  the  total 
number  of  paries 

Block  1b.  Cede  i  nter  appropriate  price 
■  inn1  (N  i  IS  only) 


.Blocks  17.-  i9.  Security  Classi fiCations.  Self- 
explanatory  ?  r-ter  U  S  Security  Classification  in 
accordance  with  U  S  Secunty  Regulations  (i  e  , 
UNCLASSIFIED)  ii  lorm  contains  classified 
•  nburmanon.  stamp  class! ficatiori  on  tfie  top  and 
i-iil :  a"i  ui  bn?  page. 


ll!o<  k  70.  intic.o  of  Abj/pa;.t  Hus  block  must 

*  *  ■  japli-if.i  '  i ;  ji ;  iiiAi  tat  ion  to  I  tip' 

men  i  '  t  ‘i-1'-  >.•> i he  lit  (unlimited)  oc.5AR  (same 


n'!  v 


■  n  -  ms  bijock  is  njecessary  if 


'be  error. i.:!  ,o  be  imtneii  y  blrmi^,  the  abstract 

!  s,  r, «, :  •  1 1  \  o d  1  o  !. i  n !  j  it?  0  o d 


Stdndard'vForm  J98  Back  (Rev  2-89) 

*  •  I  y/S  •  •  -1 '  !.  l  tt  .  S  bft  <T  70 


FOREWORD 


Opinions*  interpretations ,  conclusions  and  recommendations  are 
those  of  the  author  and  are  not  necessarily  endorsed  by  the  US 
Army. 


Where  copyrighted  material  is  quoted,  permission  has  been 
obtained  to  use  such  material. 

Where  material  from  documents  designated  for  limited 
distribution  is  quoted,  permission  has  been  obtained  to  use  the 
material. 

_  Citations  of  commercial  organizations  and  trade  names  in 

this  report  do  not  constitute  an  official  Department  of  Army 
endorsement  or  approval  of  the  products  or  services  of  these 
organizations  . 

_  In  conducting  research  using  animals,  the  investigator ( s ) 

adhered  to  the  "Guide  for  the  Care  and  Use  of  Laboratory 
Animals,"  prepared  by  the  Committee  on  Care  and  Use  of  Laboratory 
Animals  of  the  Institute  of  Laboratory  Resources,  Rational 
Research  Council  (NIH  Publication  HO.  86-23,  Revised  1985). 

_ For  the  protection  of  human  subjects,  the  investigator (s) 

adhered  to  policies  of  applicable  Federal  Law  45  CFR  46. 

_  In  conducting  research  utilizing  recombinant  DNA  technology, 

the  investigator  ( s )  adhered  to  current  guidelines  promulgated  by 
the  National  Institutes  of  Health. 

In  the  conduct  of  research  utilizing  recombinant  DMA,  the 
investigator  ( a )  adhered  to  the  HIH  Guidelines  for  Research 
Involving  Recombinant  DNA  Molecules. 

_  In  the  conduct  of  research  involving  hazardous  organisms, 

the  investigator  (s)  adhered  to  the  CDC-NIH  Guide  for  Biosafety  in 
Microbiological  and  Biomedical  Laboratories.  •  •  • 


^ A  Q c\ j. 

PI  -  Signature  ^  \  Data 


Look  and  Feel:  Haptic  Interaction 

Biomedicine 

Dr.  J.  Kenneth  Salisbury,  Jr. 
Department  of  Mechanical  Engineering  and 
Artificial  Intelligence  Laboratory 
Massachusetts  Institute  of  Technology 

Progress  Report  for  the  period 
October  1,  1994  through  September  30,  1995 
Contract  Number  DAMD17-94-C-4123 


Abstract 


for 


Accesion  For 


NTiS  CftA&l 
OTIC  TAB 
Unannounced 
Justification 


□ 


By _ 

Distribution  / 


Availability  Codes 


Dist 


B± 


Avail  and/or 
Special 


This  project  develops  and  demonstrates  key  technologies  for  advanced  vir¬ 
tual  interaction  for  surgical  simulation  find  teleoperation  systems  used  to  per¬ 
form  remote  surgical  procedures.  Our  focus  at  MIT  during  the  past  year  has 
been  to  develop  the  first  level  of  interfaces  and  algorithms  needed  to  accomplish 
the  broad  goals  of  performing  virtual  and  remote  surgical  procedures. 

During  the  past  year  we  have  implemented  a  computational  environment 
including  control  software,  graphics  software  and  computer  hardware,  to  sup¬ 
port  the  development  of  haptic  rendering  algorithms.  We  have  demonstrated 
tweezer  interaction  with  flat  compliant  surfaces  to  simulate  grasping  of  tissue 
and  developed  a  real-time,  reduced  order,  finite-element  simulation  of  compli¬ 
ant  materials.  The  resulting  system  has  been  used  successfully  to  achieve  the 
year-one  goal  of  demonstrating  multi-fingex  manipulation  of  virtual  compliant 
matP'ial.  In  addition,  we  have  performed  preliminary  investigation  and  design 
cf  a  remote  palpation  and  manipulation  device  which  will  be  used  in  the  coming 
years  to  perform  real  surgical  procedures. 
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1  Introduction 


People  use  their  hands  to  touch,  feel,  and  manipulate  the  world  around  them.  Rather 
than  studying  how  ’r  imans  achieve  their  remarkable  skill  at  haptic  interaction,  we  are 
building  systems  that  will  will  amplify,  enhance,  and  transport  these  human  skills.  We 
are  working  to  develop  technology  that  will  allow  human  users  to  touch,  feel,  grasp, 
and  manipulate  special  objects:  objects  located  remotely,  objects  existing  only  in 
simulated  worlds  (virtual  objects),  objects  too  small  or  too  large  for  normal  human 
interaction,  and  objects  that  are  too  dangerous  to  touch  with  human  hands.  We 
focus  on  how  objects  move  and  deform  in  response  to  applied  forces  and  how  objects 
feel  when  touched.  We  care  about  intrinsic  mechanical  properties  of  objects  such  as 
mass,  shape,  compliance,  texture,  and  friction  and  the  effect  of  these  properties  or  our 
perceptions  and  interactions.  Our  goal  is  to  enable  users  to  perceive  the  attributes  of 
simulated  objects  through  visual  and  haptic  modalities  and  to  permit  users  to  feel  and 
handle  objects  through  haptic  interfaces.  In  this  project  we  are  specifically  interested 
in  developing  techniques  to  permit  haptic  interaction  with  biomedical  objects  and  in 
biomedical  settings,  both  virtual  and  remote. 

Systems  for  haptic  interaction  will  need  to  have  three  parts:  a  part  that  mechan¬ 
ically  interfaces  to  the  user’s  hand  (and/or  arm),  a  part  that  determines  interactions 
between  the  hand  and  objects,  and  a  part  that  simulates  how  objects  will  behave  when 
touched.  These  are  force  feedback  devices,  haptic  algorithms,  and  physics-based  sim¬ 
ulation,  respectively.  We  are  working  to  make  major  contributions  in  all  three  areas 
by  building  a  series  of  haptic  systems  that  consist  of  a  mechanical  haptic  interface 
device,  a  real-time  computing  system,  a  dynamic  simulation  system,  and  computer 
graphics.  Already  the  system  we  have  built  provides  an  entirely  new  way  for  people 
to  interact  with  virtual  material  and  paves  the  way  for  enhanced  teleoperation  to 
perform  real  medical  procedures. 

The  work  is  being  performed  as  a  collaborative  effort  between  researchers  at  the 
MIT  AI  Lab  under  the  direction  of  Dr.  Salisbury,  and  at  Boston  Dynamics  Inc.  under 
the  direction  of  Dr.  Raibert. 

During  the  first  year,  our  goals  have  been:  to  develop  haptic  interface  tools  to 
permit  multi-finger  interaction;  to  set  up  a  development  environment  with  high  per¬ 
formance  haptic  and  graphics  facilities;  and  to  develop  a  preliminary  deformable 
media  simulation  suitable  for  tool  testing  and  into  which  real  material  properties 
(measured  by  “palpating  in”  in  the  second  year)  can  be  inserted.  To  these  ends  we 
have:  developed  tweezer  interfaces  for  the  phantom  that  can  be  operated  with  two 
fingers  and  which  sense  closure;  interfaced  a  high-speed  Pentium  via  an  ether-net 
interface  to  a  Silicon  Graphics  Indigo2  Extreme;  written  a  Pentium-based  simulation 
of  flat  compliant  material;  written  software  for  the  Pentium  to  simulate  force  interac¬ 
tions  with  a  mesh  of  compliant  material  and  communicate  deformations  to  a  visual 
graphic  model  on  the  Silicon  Graphics  workstation;  and  written  demonstrations  with 
the  compliant  material  simulators  which  permit  palpating  and  pinching  with  tweezers 
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to  simulate  bio-materials. 

2  Tweezer  Interaction  with  Flat  Surfaces 

The  Phantom  haptic  interface,  shown  in  figures  1  and  2,  was  originally  designed 
to  allow  single  point  interaction  with  simulated  materials.  The  user  would  touch 
simulated  objects  with  either  their  fingertip  attached  to  the  Phantom  through  a 
thimble,  or  through  the  tip  of  a  stylus  which  was  itself  attached  to  the  Phantom.  As 
part  of  this  first  year’',  work,  the  Phantom  hardware  was  upgraded  by  designing  a 
tweezer  interface  in  order  to  allow  grasping  of  simulated  compliant  materials. 

2.1  Tweezer  Interface  Design 

The  tweezer  interface  consists  of  a  pair  of  instrumented  leaf  spring  tweezers  which  are 
attached  to  the  passive  gimbai  of  the  Phantom  haptic  interface.  Two  iterations  on 
the  tweezer  design  were  made.  In  the  first,  a  contact  switch  was  made  which  allows 
triggering  of  the  haptics  software  when  the  tweezers  were  fully  closed.  The  second 
iteration  incorporates  strain  gauge  sensors  to  allow  partial  closure  of  the  tweezers  to 
be  measured.  The  first  version  is  shown  in  figure  3,  and  the  second  in  figure  4. 

Simple  leaf  spring  tweezers  were  chosen  due  to  their  simplicity,  light  weight,  and 
the  ease  with  which  they  could  be  attached  to  the  Phantom.  The  compliance  of  the 
tweezers  provide  a  realistic  feel  of  grasping  an  object  which  is  already  familiar  to  the 
user.  This  resulted  in  a  simple  design  with  no  additional  actuators  which  is  very  light; 
they  weigh  little  more  then  surgical  tweezers.  Finally,  the  interface  is  inexpensive  to 
produce  as  they  are  constructed  from  normal  tweezers  and  one  machined  part. 

The  first  iteration  of  the  tweezer  design  connected  one  arm  of  the  tweezer  to  the 
center  of  the  Phantom’s  quarter  gimbai.  The  line  of  action  of  all  forces  exerted  by 
the  Phantom  passes  through  the  the  center  of  this  gimbai.  With  this  first  design, 
users  always  felt  gravity  twisting  the  interface  from  between  their  fingers.  Hence,  the 
second  iteration  gravitationally  balanced  the  interface  with  respect  to  the  last  rota¬ 
tional  degree  of  freedom  on  the  gimbai  (parallel  to  the  user’s  wrist).  This  was  done 
by  separating  the  arms  of  the  tweezers  and  inserting  a  small  aluminum  bar  between 
them  which  was  in  turn  attached  to  the  Phantom  gimbai.  This  balanced  device  is 
substantially  more  comfortable  to  use.  The  strain  gauges  which  were  added  to  the 
improved  version  measure  the  separation  of  the  tweezer  leaves  and  the  grasping  force. 
Effectively,  this  is  a  simple  method  to  take  advantage  of  the  natural  tweezer  comp! 
ance  to  determine  how  hard  a  user  is  grasping.  We  have  yet  to  use  this  information  in 
the  haptics  software,  but  in  the  future  we  intend  to  measure  variable  grasping  force 
and  simulate  slipping  of  an  object  from  the  tweezer’s  grasp. 
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Figure  1:  The  Phantom  haptic  interface. 

2.2  Tweezer  Interface  Software 

The  current  tweezer  demonstration  software  models  a  uniform  flat  surface  (tissue) 
that  may  be  “plucked”  and  stretched,  much  like  the  skin  on  the  back  of  one’s  hand. 
When  the  tweezers  are  placed  beneath  a  plane  corresponding  to  the  surface,  they 
are  repelled  upward  via  a  simulated  linear  spring.  If,  while  beneath  the  plane,  the 
tweezers  are  closed,  the  simulation  acts  as  if  the  surface  has  been  grasped  at  a  point. 
The  user  may  then  stretch  the  surface,  feeling  a  simulated  linear  spring  which  connects 
them  to  the  point  where  the  surface  was  plucked.  If  the  tweezers  are  closed  and 
they  come  into  contact  with  the  virtual  surface,  they  may  skate  freely  across  the 
surface.  To  further  demonstrate  the  variance  in  surface  properties  which  may  be 
simulated,  the  compliance  of  the  surface  varies  from  left  to  right  across  the  surface. 
Elementary  computer  graphics  also  accompany  the  simulation  in  order  to  aid  the  user 
in  visualizing  the  surface  deformation. 

The  simulation  models  linear  springs  primarily  because  they  provide  a  large  range 
of  travel  under  the  modest  output  force  restrictions  of  the  Phantom.  Models  using 
arbitrary,  non-linear  virtual  springs  are  trivial  to  create,  however,  we  have  found  that 
users  have  an  easier  time  visualizing  the  surface  when  they  can  stretch  it  a  substantial 
distance  (e.g.  2”  instead  of  3/4”),  and  this  corresponds  well  with  the  linear  spring 
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Figure  2;  Schematic  of  Phantom  haptic  interface  (Courtesy  of  David  Brock,  MIT 
Artificial  Intelligence  Laboratory). 

model  and  with  the  Phantom’s  range  of  motion. 

The  illusion  perceived  by  the  user  is  similar  to  that  of  plucking  a  sheet  of  saran 
wrap,  or  as  mentioned  above,  plucking  the  skin  on  the  back  of  and  simulone’s  hand. 
This  is  accompanied  by  simple  graphics,  which  show  the  contour  of  such  a  surface  as 
it  is  deformed, 

3  Haptic  Interaction  with  Deformable  Objects 

As  part  of  this  first  year’s  work,  an  interactive  deformable  media  simulation  (IDMS) 
was  developed,  which  allows  more  general  simulation  of  deformable  media  than  the 
tweezer  interface  software  described  above.  This  work  resulted  in  an  MIT  Master  of 
Science  Thesis  which  is  included  as  Appendix  B  for  reference.  The  following  is  a  brief 
description  of  the  system. 

The  IDMS  system  provides  real-time  visual  and  force  feedback  by  means  of  a 
graphics  display  and  the  Phantom  haptic  interface.  A  finite  element  modeling  ap¬ 
proach  is  used  to  represent  structural  and  viscoelastic  properties  and  to  simulate 
dynamic  behavior.  Media  is  represented  as  a  discrete  network  of  masses  and  con¬ 
stitutive  mechanical  elements,  such  as  springs  and  dashpots.  In  order  to  maintain 
real-time  simulation  speed,  the  system  (a)  uses  a  surface  representation,  as  opposed 
to  a  volumetric  representation,  that  is  coupled  with  a  surface  restoring  force  and  (b) 
conserves  computation  power  by  only  rendering  media  local  to  the  interaction  point. 
Haptic  interaction  forces  that  satisfy  causality  and  passivity  are  generated  using  sim- 
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Figure  3:  Instrumented  tweezers  attached  to  Phantom  gimbal.  Binrry  contact  in¬ 
formation  is  provided  through  an  electrical  switch  that  is  closed  when  both  tweezer 
sides  are  touching. 

pie  collision  detection  techniques  based  on  point  intersections.  The  interaction  tasks 
that  were  investigated  include  palpation,  stroking,  and  plucking  and  give  a  good  sense 
of  realism.  The  primary  features  of  Interactive  Deformable  Media  Simulation  (II)MS) 
are  described  below. 

3.1  Modelling  Approach 

The  modeling  approach  used  in  IDMS  to  capture  properties  of  virtual  objects  utilizes 
a  c  .mbination  of  finite  element  modeling  (FEM)  theory  and  surface  encapsulation 
of  volumetric  properties  to  convincingly  convey  the  feeling  of  deformation  to  the 
user.  Each  object  model  is  comprised  of  a  two-dimensional  layer  of  finite  tetrahedral 
structural  elements,  networked  together  to  form  the  surface  of  the  deformable  model, 
as  shown  in  Figure  5.  Tetrahedral  elements  are  used  because  they  are  structurally 
stable  and  have  low  mass  density,  requiring  minimal  discrete  elements  per  unit  volume 
to  represent  them.  The  encapsulation  of  volumetric  effects  is  achieved  by  adding  a 
restoring  force  between  the  top  surface  of  the  tetrahedral  mesh  and  the  equilibrium 
position  of  the  top  surface.  The  user  only  interacts  with  the  top  surface  of  the 
mesh,  both  visually  and  haptically.  The  notion  of  a  haptic  window  is  introduced  to 
allow  simulation  of  object  dynamics  local  to  the  interaction  point.  Only  media  that 


Figure  4:  Improved  Tweezer  Interface 


falls  within  this  geometrically  shaped  window  is  simulated  in  order  to  generate  real¬ 
time  response.  Combined,  these  features  serve  to  haptically  and  visually  display  the 
significant  inodes  of  deformation  to  the  user. 

A  discrete  simulation  model  (DSM)  is  chosen  to  represent  the  surface  mesh  de 
scribed  above.  A  DSM  is  a  discretized  network  of  lumped  parameter  elements  and 
atoms  that  approximates  the  physical  behavior  of  a  continuum  media.  In  the  present 
context  of  deformable  objects,  atoms  are  point  masses  connected  to  neighboring  atoms 
through  constitutive  mechanical  elements.  Each  tetrahedral  element  is  represented 
by  four  link  elements  and  four  bounding  atoms,  shared  with  neighboring  tetrahedra. 
The  DSM  is  stored  in  graph  form  to  facilitate  alteration  of  the  model  topology.  Sim¬ 
ulation  of  the  DSM  state  is  accomplished  by  a  multi-step  algorithm  that  is  evaluated 
within  the  topology  of  the  model.  During  each  time  step,  forces  are  summed  for  each 
mobile  atom  based  on  external  forces  and  internal  forces  from  neighboring  elements. 
The  two  state  variables  of  position  and  velocity  for  each  atom  are  then  numerically 
integrated  resulting  in  the  DSM  state  for  the  next  time  step. 

The  restoring  force  between  the  top  layer  of  the  mesh  and  the  equilibrium  position 
is  achieved  within  the  framework  of  the  DSM.  Essentially,  a  massless  immobile  home 
atom  is  added  at  the  equilibrium  position  of  each  atom  located  on  the  surface  of  the 
mesh.  A  connecting  home  element  is  added  between  the  surface  atom  and  its  re¬ 
spective  home  atom.  As  the  virtual  mesh  is  deformed  interactively,  the  superposition 
of  forces  produced  by  home  elements  in  tension  compactly  simulates  the  volumetric 
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Figure  5:  A  fragment  of  media  represented  by  a  two-dimensional  layer  of  tetrahedral 
elements  (dashed  line).  Volumetric  effects  are  simulated  by  applying  a  restoring  force 
between  the  top  surface  (bold  line)  and  the  equilibrium  position  of  the  top  surface. 

compression  of  the  continuum  media.  Sequences  showing  mesh  deformation  during 
finger  palpation,  and  during  plucking  are  given  in  figures  6  and  7  respectively.  Please 
see  Chapter  6  of  Appendix  B  for  a  more  complete  description  of  these  results. 

3.2  Haptic  “Windowing” 

Current  haptics  technology  primarily  involves  environment  interaction  with  fingers 
or  tools  that  have  localized  mechanical  effect.  Just  as  a  graphics  simulation  only 
renders  those  objects  that  fall  within  the  viewing  window  of  the  monitor,  a  haptics 
simulation  need  only  render  those  effects  that  are  spatially  local  to  the  interaction 
point  to  minimize  computation  effort.  This  haptic  “window”  is  not  as  spatially  rigid 
as  the  rectangular  viewing  space  of  a  traditional  graphics  display  because  physical 
elements  throughout  the  model  contribute  some  work  transfer  to  the  user.  Thus,  the 
haptic  window  must  selectively  include  enough  elements  to  the  point  of  diminishing 
marginal  return  for  computational  cost  versus  perceptual  realism.  Local  simulation  is 
achieved  in  IDMS  by  only  simulating  media  that  lies  within  a  sphere  centered  at  the 
interaction  point;  the  rest  of  the  model  is  immobilized  and  hence  not  unnecessarily 
simulated  during  each  time  step.  The  size  of  this  sphere  is  empirically  chosen  such 
that  stroking  along  the  surface  of  the  virtual  object  is  hapiically  smooth. 

The  subsystem  of  IDMS  that  calculates  the  interaction  force,  the  master  interac¬ 
tion  object  (MIO),  is  the  software  element  connecting  the  simulation  and  the  haptic 
interface.  The  MIO  models  in  software  the  contact  interaction  between  the  human 
and  the  object  surface.  The  MIO  inputs  position  information  from  the  haptic  in 
terface  and  the  DSM  simulation  and  outputs  a  force  signal  back  through  the  haptic 
interface  and  the  DSM  simulation.  Each  haptic  interface  that  is  connected  to  IDMS 
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is  assigned  a  different  MIO. 

Specifically,  the  MIO  is  defined  by  a  static  geometric  shape  that  is  dynamically 
centered  at  the  endpoint  of  the  haptic  interface.  This  shape  represents  the  geometry 
of  the  real-world  tool  being  modeled  for  interaction.  During  contact,  atoms  that 
are  positioned  internal  to  the  geometric  boundary  of  the  MIO  receive  an  external 
force.  A  reaction  force  is  applied  to  the  haptic  interface  equal  to  the  negative  sum  of 
external  forces  applied  on  DSM  atoms  by  the  MIO.  Currently,  each  atom  receives  a 
force  proportional  to  its  internal  penetration  distance  normal  to  the  geometric  surface, 
simulating  an  elastic  contact.  A  simple  collision  detection  algorithm  is  used  to  find 
those  atoms  of  the  model  that  are  internal  to  the  MIO  boundary.  Collision  detection 
with  the  MIO  is  computationally  efficient  because  only  point  intersections  need  to  be 
found.  Increasingly  complex  geometries  can  be  created  for  the  MIO  as  a  function  of 
processing  power  and  more  efficient  collision  detection  algorithms. 


4  Software  and  Hardware  Specifications  for  Hap¬ 
tic  Interaction 

One  of  the  primary  accomplishments  during  this  past  year  was  the  development  of 
an  Interactive  Deformable  Media  Simulation  (IDMS).  As  part  of  this  effort,  several 
high-performance  components  were  assembled  to  form  hardware  architecture  that 
IDMS  runs  upon.  Each  of  the  major  hardware  components  and  their  connectivity  is 
described  below.  Please  see  Chapter  5  of  Appendix  B  for  a  more  thorough  description 
of  this  implementation. 

The  haptic  computational  engine  is  responsible  for  running  processes  that  simulate 
the  dynamics  of  the  virtual  environment  (environment  process),  servo  the  Phantom 
haptic  interface  (haptic  process),  and  update  the  visual  process  on  the  visual  engine 
with  the  current  environment  state.  The  haptic  and  environn.  „nt  processes  are  run 
as  one  combined  process  because  of  the  inherent  interactive  coupling  between  the 
environment  dynamics  and  the  haptic  interface.  A  120  MHz  Pentium-based  computer 
running  Linux,  a  unix-based  multitasking  operating  system,  is  used  to  serve  these 
purposes. 

The  visual  engine  is  responsible  for  displaying  the  state  of  the  virtual  environment 
at  a  sufficient  refresh  rate  to  the  human  visual  system.  A  200  MHz  Silicon  Graphics 
Indigo  II  Extreme  workstation,  used  as  the  IDMS  visual  engine,  is  designed  for  fast 
graphics  computation  because  its  architecture  has  an  integrated  graphics  pipeline  for 
processing  low-level  graphics  routines.  The  visual  processes  for  IDMS  use  the  Silicon 
Graphics  Openlnventor  library  toolkit  built  upon  the  OpenGL  protocol  for  platform 
independent  graphics  rendering.  Incorporated  into  the  standard  viewer  with  this 
library  are  basic  viewing  primitives  such  as  zooming,  panning,  rotating,  lighting,  and 
shading  that  are  easily  controllable  by  the  user.  Lastly,  data  is  communicated  between 
the  haptic  and  visual  engines  over  Ethernet  using  a  UDP  packet  level  connection, 


ensuring  high  throughput  rates  of  data  transfer. 

The  choice  of  hardware  architecture  described  above  is  heavily  influenced  by  the 
choice  of  concurrent  run-time  software  model  used  in  simulating  IDMS.  Under  the 
concurrent  model,  separate  operations  in  the  environment  run  in  parallel  on  different 
processors.  In  IDMS  two  main  processes,  which  handle  three  discrete  subprocessee, 
run  concurrently  on  the  haptic  and  visual  engines.  The  client  process,  running  on 
the  haptic  engine,  is  comprised  of  the  environment  and  haptic  processes  and  updates 
at  approximately  1  kHz.  The  server  process,  which  runs  on  the  visual  engine  and 
is  equivalent  to  the  visual  process,  receives  environment  data  from  the  client  process 
and  renders  the  visual  state  of  the  environment  at  approximately  30  Hz.  In  order  to 
decouple  loading  effects  between  the  client  and  server  processes,  a  separate  shared- 
memory  process  runs  on  both  the  haptic  and  visual  engines  to  handle  interprocessor 
data  communication.  The  client  process  updates  the  shared-memory  procesr  on  the 
haptic  engine  with  differential  changes  in  environment  data,  hence  minimizing  latency 
in  the  time-critical  client  loop.  The  two  individual  shared-memory  processes  run 
synchronously  using  a  hand-shaking  protocol  and  serve  to  transfer  the  entire  visual 
state  of  the  environment  at  30  Hz.  Lastly,  the  server  process  runs  synchronously  with 
the  shared-memory  process  on  the  visual  engine  and  hence  also  updates  the  screen 
at  30  Hz. 

5  Multi-Finger  Manipulation 

With  the  development  of  sensored  tweezers  and  the  implementation  of  the  IDMS 
software  described  above,  it  became  a  nearly  trivial  extension  to  modify  the  software 
to  include  a  second  master  interaction  object  which  could  simultaneously  push  on  the 
surface.  A  second  nearly  trivial  extension  to  the  code  permitted  the  tweezer  tips  to 
become  coupled  to  the  nearest  node  upon  closing  them,  thereby  permitting  both  of 
the  tweezers  to  pull  or  push  on  the  mesh.  This  enabled  users  to  prod,  grasp  and  tug 
upon  the  compliant  material,  deforming  it  by  palpation,  pulling  and  stretching.  The 
results  of  these  new  capabilities  are  illustrated  in  the  video  companion  to  this  report, 
entitled  “Look  and  Feel:  Haptic  Interaction  for  Biomedicine.  Video  Companion  to 
the  Progress  Report  for  October  1,  1994  thru  September  30,  1995,  ARPA  Contract 
Number  DAMD17-94-C-4123” 

6  Work  in  Progress  and  Plans  for  Year  2 

While  the  Phantom  has  a  reasonable  workspace  for  operations  in  which  the  user’s 
forearm  is  stationary,  it  is  desirable  to  permit  a  larger  range  of  motion  for  some  pro¬ 
cedures.  To  this  end  we  will  be  developing  a  larger  workspace  interface,  either  by 
adapting  the  three-times  larger  “Tool-Handle”  interface  or  by  adding  additional  mo¬ 
tion  freedoms  to  the  Phantom.  To  help  validate  basic  interactions  via  newly  developed 


interface  tools  (tweezers,  scalpel,  perhaps  forceps)  we  will  continue  to  enhance  the 
IDMS  compliant  material  simulator.  This  will  be  accomplished  by  investigating  the 
variation  of  mesh  parameters  to  simulate  non-homogeneous  materials  (e.g.  flesh  with 
underlying  bone,  inclusions,  tumors,  etc.),  speeding  up  the  algorithm  to  enable  larger 
objects  and  forming  organ-like  compliant  geometries.  We  will  also  begin  gathering 
material  property  data  (e.g.  local  stiffness  and  viscosity)  by  using  a  phantom  to  sys¬ 
tematically  probe  real  compiant  material  (such  as  physical  mockups  of  bio-material 
or  animal  parts).  The  data  will  be  validated  with  the  IDMS  system  and  prepared  for 
transfer  to  the  compliant  Tenderer  being  developed  by  Boston  Dynamics,  Inc.  under 
companion  funding  from  ARPA. 

Our  experience  in  this  first  year’s  work  and  in  past  haptic  interface  and  robotic 
design  suggests  that  a  properly  designed  master/slave  teleoperation  system  may  over¬ 
come  many  of  the  limitations  of  current  endoscopic  surgery,  and  possibly  allow  more 
open  procedures  to  be  converted  to  endoscopic  ones.  The  two  problems  we  see  in 
current  endoscopic  instruments  is  a  lack  of  mobility  and  a  lack  of  force  and  tactile 
sensation  when  compared  to  open  surgical  procedures.  In  order  to  pursue  how  we  may 
address  these  problems,  we  have  begun  to  investigate  possible  designs  for  multi-dof 
master-slave  teleoperation  system  to  be  used  for  endoscopic  surgery.  The  resulting 
hardware  will  provide  the  basis  for  developing  advanced  surgical  teleoperation  tech¬ 
niques  in  the  3rd  year.  During  this  coming  year,  we  intend  to  study  the  feasibility  of 
building  a  system  which  is  sensitive  enough  to  feel  soft  tissue,  while  ultimately  giv¬ 
ing  a  surgeon  increased  mobility,  and  the  ability  to  work  through  the  small  incisions 
required  of  endoscopic  surgery.  To  start,  we  intend  to  build  a  simple,  low  degree  of 
freedom,  force- reflecting  prototype  to  be  used  to  study  remote  palpation  of  tissue.  We 
intend  to  continue  with  our  successful  approach  of  designing  very  lightweight,  rigid, 
very  low  friction  devices  in  order  to  accurately  transmit  force  information. 


7  Conclusions 

Our  first  year’s  effort  has  been  successful  in  its  goal  of  extending  our  ability  to  model 
and  haptically  interact  with  simulated  biomaterials. 

We  have  developed  a  new  tweezer  interface  to  the  Phantom  system  which  permits 
several  fingers  to  be  used  in  the  manipulation  of  simulated  objects.  This  interface 
provides  the  user  with  a  familiar  tool  that  closely  approximates  the  feel  experienced 
in  day-to-day  medical  practice. 

While  real-time  simulation  of  complex  non-homogeneous  materials  is  a  challeng¬ 
ing  computational  problem,  we  have  found  a  simplified  approach  which  permits  us 
to  rapidly  compute  the  deformations  and  forces  encountered  while  probing  a  sheet  of 
visco-elastic  material.  By  building  essentially  a  “thick”  surface  representation  com¬ 
posed  of  tetrahedral  units  we  are  able  to  reduce  the  computation  time  while  retaining 
a  reasonably  rich  representation  of  material  properties.  In  the  mesh,  the  nodes  are 
assigned  mass  values,  and  springs  and  dampers  connect  these  nodes,  in  a  regular  man- 
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ner,  to  approximate  the  material  properties.  The  mesh  itself  may  be  free  to  move 
in  space  or  some  of  the  nodes  may  be  tied  to  a  rigid  substrate  by  additional  springs 
and  dampers.  The  effect  enables  us  to  simulate  both  free  standing  tissue  samples  and 
tissue  attached  via  connective  tissue  to  rigid  bone  structure.  Though  not  utilized  yet, 
it  is  important  to  note  that  the  constitutive  functions  may  be  non-linear  -  permitting 
non-linear  spring  and  viscosity  forces  to  act  within  the  model. 

Taken  together  these  developments  provide  a  firm  foundation  from  which  to  ad¬ 
dress  the  second  and  later  year  goals  of  our  work.  The  fact  that  we  easily  added 
multi-hand  (2  tweezers)  interaction  and  grasping  capability  to  the  system  is  a  mea¬ 
sure  of  the  systems  extensibility.  In  the  near  future  we  plan  to  address  issues  of 
making  the  simulation  more  fine  grained  and  efficient  by  algorithmic  improvement 
and  perhaps  by  utilization  of  multi-processor  computation.  The  organization  of  the 
syi  tem  permits  both  arbitrary  specification  of  the  rest  shape  of  the  compliant  mate¬ 
rial  and  variation  in  the  mass-spring-damper  elements  across  the  surface.  This  will 
enable  us  to  model  more  biomedically  relevant  shapes  and  include  parameters  rep¬ 
resenting  real  material  property  values.  We  also  expect  that  the  addition  of  mesh 
altering  actions  such  as  cutting  and  piercing  will  be  able  to  be  incorporated  in  the 
coming  year. 


A  Appendix:  Communications  and  Visual  Dis¬ 
play  Summary 

The  following  summarizes  work  done  to  improve  PC/SGI  communcations  and  SGI 
visual  display.  Below,  a  more  detailed  review  is  given. 

1.  Communications  between  the  PC  and  SGI  were  optimized  for  speed.  Multiple 
processes  were  created  to  facilitate  introduction  of  a  real-time  OS. 

2.  Shared  memory  was  implemented  on  the  PC  and  SGI  as  a  common  database 
of  environment  information  for  I/O  processes  and  their  parents  to  share. 

3.  A  first  look  was  taken  at  improved  visual  rendering  of  a  physical  (haptic) 
database  with  limited  data  points.  For  example,  B-Spline  approximations  to  the 
IDMS  surfaces  of  only  a  few  real  points;  texture  mapping  of  pictures  onto  ob¬ 
jects  (purely  cosmetic);  creating  a  background  scene  of  non-interactive  objects 
that  will  be  displayed  on  the  graphics  side  but  not  simulated. 

A.l  Communications  Between  the  PC  and  SGI 

A  UDP/IP  socket-based  communications  link  was  uet  up  between  linux-based  PC’s 
and  UNIX-based  SGI’s.  This  replaced  the  old  RPC  (remote  procedure  call)  interface, 
which  was  slowing  things  down  by  requiring  an  SGI  response  for  every  data  packet 


sent  by  the  PC.  With  the  UDP  method,  data  packets  are  sent  at  regular  intervals 
(called  FRAME  packets)  to  update  the  graphics  frame  currently  displayed  by  the 
SGI.  Additional  packets  may  be  sent  for  creating  objects,  changing  colors,  texture 
mapping,  etc.,  similar  to  the  old  RPC  routines. 

Central  to  the  PC  side  are  the  function  calls  in  D Viewer-call s.h,  and  DViewer.calls.c 
the  header  file  must  be  included  in  one’s  PC  code,  and  one  must  link  the  library 
/projects/ wam/dave/dviewer/lib/libDViewer-linux. a 

The  UDP  method  does  not  necessarily  guarantee  receipt  of  packets.  It  has  precious 
little  error- checking  in  terms  of  packet  resends;  i.e.,  the  PC  will  spit  out  frame  updates 
at  30Hz,  and  the  SGI  will  catch  and  display  as  many  as  it  can,  but  if  the  SGI  misses 
one,  it  floats  off  into  the  ether(net)  and  dies.  This  ensures  that  frames  don’t  sit 
in  the  SGI  input  queue  and  get  displayed  hundreds  of  milliseconds  after  they  were 
generated.  The  SGI  will  just  catch  the  next  packet  and  display  it,  keeping  relatively 
real-time  w.r.t.  the  haptic  process.  One  caveat  is  thus:  if  you  design  new  packets 
that  must  be  sent  over  and  MUST  be  received  mid  stream,  you  need  to  create  an 
SGI-iPC  reply  and  corresponding  PC  timeout  if  reply  is  not  received  to  resend  the 
packet. 


A. 2  Dual  Processes 

Both  the  SGI  and  PC  have  parent  processes,  which  do  the  main  task,  and  child 
processes  to  perform  the  I/O  tasks.  This  was  initially  designed  to  compartmentalize 
tasks  and  make  it  easier  for  the  IDMS  code  to  run  REALLY  fast  without  having  to 
stop  for  I/O  all  the  time.  The  child  I/O  process  takes  about  1  ms  to  send  a  packet; 
this  ;s  actually  a  fair  bit  of  time  if  it  controls  the  processor  for  that  1  ms  every  33  ms 
(30Hz).  The  dual-process  architecture  would  enable  the  processor  to  give  the  child 
process  just  enough  timeslices  to  get  a  packet  sent,  in  a  single  33-ms  loop;  the  general 
idea  is  that  the  parent  code  runs  full-speed,  and  only  gets  superseded  every  ms  or  so 
for  a  few  clock  cycles  to  be  devoted  to  I/O. 

In  practice,  Linux  is  NOT  a  real-time  OS,  and  doesn’t  do  real-time  scheduling 
of  processes;  therefore  there  was  much  difficulty  in  getting  the  system  to  run  as 
planned.  As  such,  the  current  implementation  actually  does  pass  lOOoften  the  user 
calls  IViewer_begin_update)  by  setting  a  semaphore  that  wakes  the  child  up.  Porting 
to  LynxOS  would  solve  this  problem. 

A. 3  Shared  Memory 

To  create  a  data  path  between  the  parent  processes  (PC-haptics,  SGI-graphics)  and 
the  I/O  processes  for  each  computer,  shared  memory  arenas  have  been  allocated  on 
both  the  PC  and  the  SGI. 

The  SGI  implementation  is  much  better,  and  it  is  (by  no  means  urgent  or  even 
high-priority)  wishful  task  to  implement  the  SGI  version  on  the  PC,  but  they  both 


work.  The  SGI  divides  shared  memory  into  two  segments:  command-space  shared 
memory,  and  data-space  shared  memory. 

In  command-space  shared  memory,  the  child  I/O  process  informs  the  graphics 
parent  of  commands  it  has  received  and  needs  to  tell  the  parent  about  (such  as 
receiving  an  IViewer_texture()  command  with  a  filename  argument.  The  child  will 
then  write  the  command  to  memory,  as  well  as  the  arguments  (filename).  When  the 
graphics  parent  goes  through  its  Xt  callback  every  33ms  or  so  (set  by  the  user),  it 
processes  the  command  and  resets  a  flag  allowing  the  child  to  write  a  new  command. 

Data-space  shared  memory  is  used  to  communicate  the  current  positions  of  graph¬ 
ical  objects  to  the  parent  graphics  process.  When  the  child  I/O  process  receives  a 
data  packet  containing  position/orientation  information,  it  writes  it  to  indexed  po¬ 
sitions  in  shared  memory;  the  parent  graphics  process  will  then  read  through  this 
data-space  shared  memory  before  every  screen  update  (frame  refresh). 

A  similar  system  is  in  use  on  the  Linux  side,  except  that  separate  shared  memory 
segments  are  allocated  for  every  object,  instead  of  indexing  one  large  shared  memory 
segment.  The  parent  haptics  process  is  responsible  for  writing  current  data  to  shared 
memory  with  I  Viewer  jset  _config-shm( )  and  IViewer_move_vertex-shm();  the  I/O  child 
then  reads  through  this  on  an  IViewer_begin_update()  and  sends  a  packet  to  the  SGI. 

A. 4  Improved  Visual  Display  Techniques 

The  SGI  machines  have  graphics  capabilities  which  extend  far  beyond  simply  dis¬ 
playing  the  collection  of  points  that  the  haptics  platforms  are  simulating.  With  this 
in  mind,  and  the  fact  that  increased  visual  feedback  and  realism  is  a  good  thing,  it 
should  be  possible  to  improve  fidelity  on  the  visual  rendering  side  cf  a  demo  without 
interfering  with  the  stability  of  the  haptics  software.  Several  ideas  in  this  direction 
have  been  developed  or  are  in  development: 

A. 4.1  Texture  Mapping 

The  Indigo2  Extreme  platform  cannot  usefully  texture-map  a  scene.  It  can  perform 
the  task,  i.e.,  create  a  nice  picture,  but  cannot  texture  a  simple  bitmap  onto  a  simple 
cube  at  the  i  10  Hz  frame  rates  necessary  for  decent  visual  animation.  The  fea¬ 
ture  IViewer.textureQ  has  been  added  to  the  SGI  library  of  functions,  but  it  is  not 
extremely  useful  unless  performed  on  a  much  faster  machine,  i.e.  an  Onyx/Reality 
Engine/ Impact. 

A. 4. 2  B-Splines 

B-splines  are  a  good  idea  for  better  approximation  of  a  smooth  surface  being  simulated 
haptically  using  only  a  few  scattered  points.  They  eliminate  surfaces  looking  like 
patchwork  polygons.  We  have  only  implemented  Bezier  surfaces  with  the  control 
point/knot  vector  interface  that  Inventor  has.  This  suffers  from  the  problem  that 


the  surface  doesn’t  intersect  the  control  points,  only  gets  close  to  them,  so  one  is  not 
really  representing  reality.  We  may  have  to  develop  our  own  B-spline  generator. 

A.4.3  Non-Interactive  Environments 

Non-intcractive  environments  are  very  good  idea.  Setting  up  a  convincing  visual 
background,  possibly  in  Inventor  file  format  (”.iv”)  would  be  a  great  way  to  improve 
realistic  scenes,  as  long  as  one  does  not  need  to  interact  with  the  objects.  Interacting 
with  the  objects  would  require  more  code.  The  idea  of  using  something  like  the 
sgi  demo  /usr/share/src/Inventor/demos/SceneViewer  to  do  world  creation/building 
may  be  good. 

B  Appendix:  Haptic  Interaction  with  Deformable 
Objects  Using  Real-Time  Dynamic  Simulation 

The  following  thesis  gives  a  detailed  description  of  the  modelling,  mathematics,  and 
software  behind  the  Interactive  Deformable  Media  Simulation  described  in  this  report, 
in  addition  to  more  detailed  results  and  demonstrations  of  its  use. 
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Abstract 

This  thesis  discusses  the  design  and  implementation  of  an  interactive  deformable  media  sim¬ 
ulation  (IDMS)  that  provides  real-time  visual  and  force  feedback  by  means  of  a  graphics 
display  and  haptic  interface.  A  finite  element  modeling  approach  is  used  to  represent  struc¬ 
tural  and  viscoelastic  properties  and  simulate  dynamic  behavior.  Media  is  represented  as  a 
discrete  network  of  masses  and  constitutive  mechanical  elements,  such  as  springs  and  dash- 
pots.  Real-time  simulation  is  achieved  by  (a)  using  a  surface  representation  coupled  with 
a  surface  restoring  force  to  encapsulate  volumetric  properties;  (b)  conserving  computation 
power  by  rendering  media  local  to  the  interaction  point;  and  (c)  evaluating  the  simulation 
within  the  topology  of  the  model.  Haptic  interaction  forces  that  satisfy  causality  and  passivity 
are  generated  using  simple  collision  detection  techniques  based  on  point  intersections.  Res¬ 
ults  of  fundamental  interaction  tasks  including  palpation,  stroking,  and  plucking  are  discussed 
within  the  IDMS  framework  and  are  shown  to  be  highly  representable  of  real  interactions. 

Thesis  Supervisor:  J.  Kenneth  Salisbury 
Title:  Principal  Research  Scientist 
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Chapter  1 

Introduction 


1.1  Background 

“ Jennifer  Roberts,  the  mother,  is  training  to  become  a  surgeon  and  is  at  her 
[Virtual  Environment]  station  studying  past  heart  operations. 

She  previously  spent  many  hours  familiarizing  herself  with  the  structure  and 
function  of  the  heart  by  working  with  the  virtual-heart  system  she  acquired  after 
deciding  to  return  to  medical  school  and  to  specialize  in  heart  surgery.  This  sys¬ 
tem  includes  a  special  virtual-heart  computer  program  obtained  from  the  National 
Medical  library  of  physical/Computational  Models  of  Human  Body  systems  and 
a  special  haptic  interface  that  enables  her  to  interact  manually  with  the  virtual 
heart.  Special  scientific  visualization  subroutines  enable  her  to  see,  hear,  and  feel 
the  heart  (and  its  various  component  subsystems)  from  various  vantage  points 
and  at  various  scales.  Also,  the  haptic  interface  which  includes  a  special  suite  of 
surgical  tool  handles  for  use  in  surgical  simulation  analogous  to  the  force-feedback 
controls  used  in  advanced  simulations  of  flying  or  driving),  enables  her  to  prac¬ 
tice  various  types  of  surgical  operations  on  the  heart.  As  part  of  this  practice, 
she  sometimes  deliberately  deviates  from  the  recommended  surgical  procedures  in 
order  to  observe  the  effects  of  such  deviations - ”[5,  p25] 

Virtual  environments,  as  exemplified  in  the  vision  above,  are  systems  that  integrate  com¬ 
puters,  man-machine  interfaces,  and  human  operators  to  synthesize  perceptual  immersion  into 
artificial  reality.  These  systems  have  the  potential  to  allow  humans  to  create  design  proto¬ 
types  quickly  and  inexpensively;  to  train  in  complex,  hazardous  environments;  to  operate  on 
remote  tasks  through  transmission  of  sensorimotor  signals;  and  create  artificial  environments 
that  are  physically  unrealizable.  Clearly,  the  environment  being  modeled  must  be  accurately 
represented  in  order  to  be  perceptually  acceptable  as  a  substitute  for  reality.  Furthermore, 
understanding  of  the  human  psychophysical  system  is  necessary  since  virtual  environments 
transfer  information  to  and  from  the  human  operator  through  sensorimotor  signals.  These 
requirements  impose  strict  constraints  on  the  bandwidth,  fidelity,  and  non-obtrusiveness  of 
the  virtual  environment.  With  continuously  increasing  growth  in  computational  speed  and 
hardware,  however,  virtual  environments  are  becoming  feasible  as  powerful  tools  to  assist 
humans. 
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An  important  modality  of  virtual  environments  that  has  received  little  attention  relative  to 
visual  and  auditory  forms  of  feedback  is  haptics ,  or  the  touch  channel.  Haptics  is  unique  from 
vision  and  audition  because  both  sensing  and  actuation  occur  on  the  environment.  The  ability 
to  haptically  explore  and  manipulate  objects  greatly  enhances  the  human’s  sense  of  immersion 
in  a  virtual  environment  [5].  A  synergistic  effect  occurs  when  visual,  auditory,  and  haptic 
cues  are  combined  and  presented  to  the  user.  This  coupling  between  different  sensorimotor 
channels  is  meaningfully  exploited  in  software  to  facilitate  and  enhance  representation  of 
multi-modal  scenes. 

Advances  in  haptic  interaction  have  been  primarily  limited  by  the  performance  of  haptic 
interfaces.  Haptic  interfaces  typically  are  electromechanical  devices  that  interface  with  the 
user’s  hand  or  finger  and  transmit  position  and  force  information  bidirectionally  between  the 
human  and  the  virtual  environment.  Programming  the  force  behavior  of  the  haptic  interface 
as  a  function  of  position  is  equivalent  to  controlling  the  mechanical  impedance  felt  by  the 
user  [8].  Design  constraints  on  haptic  interfaces  are  motivated  to  achieve  psychophysical 
transparency  between  the  desired  programmed  behavior  of  the  virtual  environment  and  the 
human  mechano-sensorimotor  system.  Inertia  and  friction,  for  example,  in  a  haptic  interface 
only  serve  to  corrupt  the  transmission  of  sensorimotor  signals  and  reduce  fidelity.  The  device 
used  in  this  thesis,  the  PHANToM  haptic  interface  [11],  is  a  high-performance  ground-based 
haptic  interface  with  three  active  translational  and  three  passive  rotational  degrees  of  freedom. 
The  PHANToM  uses  a  lightweight  cable-driven  parallel  linkage  to  transmit  haptic  signals 
to  and  from  the  human  fingertip  (Section  5.1.2).  Because  of  the  high  bandwidth  and  fidelity 
exhibited  by  the  device,  the  Phantom  succeeds  in  satisfying  the  transparency  requirement. 

In  addition  to  hardware,  an  intrinsic  part  of  a  modeled  virtual  environment  is  the  software 
representation  and  simulation  of  objects  that  comprise  it.  Three  major  components  of  a 
dynamic  environment  simulation  are  listed  below. 

1.  Dynamics  simulation. 

2.  Collision  detection. 

3.  Contact  force  modeling. 

Haptic  display  is  primarily  concerned  with  generation  of  interaction  forces  (item  3  above). 
Because  visual  and  haptic  display  of  object  characteristics  share  similar  attributes,  some 
mathematical  techniques  from  the  established  field  of  computer  graphics  [6],  for  example 
collision  detection  (item  2),  can  be  borrowed  to  provide  insight  into  methods  for  haptic  dis¬ 
play.  Although  parallels  between  visual  and  haptic  simulation  exist  ,  the  high  bandwidth  of 
the  human  haptic  system  requires  a  considerably  faster  display  of  information,  particularly 
at  the  local  areas  of  contact  [5],  Furthermore,  many  significant  mechanical  attributes  such 
as  surface  friction,  texture,  and  impedance  are  not  appropriately  addressed  in  the  frame¬ 
work  of  graphics.  Thus,  haptic  display  necessitates  a  fundamentally  separate  perspective  for 
simulation  design. 

A  coherent  framework  for  haptically  representing  shape,  surface  properties,  and  bulk 
properties  of  rigid  objects  is  discussed  in  [21].  Interaction  force  algorithms  have  been  de¬ 
veloped  for  rigid  polyhedra  that  are  based  on  static  geometry  constraints.  The  perception 
of  a  rigid  surface  is  conveyed  if  the  programmable  stiffness  is  made  high  relative  to  the  hu¬ 
man’s  proprioceptive  sense  for  discerning  stiffness.  A  point-based  paradigm  is  described 
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in  [26,  27]  which  computes  interaction  forces  based  on  Hooke’s  law  when  virtual  contact  is 
made.  Another  constraint-based  display  system  utilizes  edge  contact  models  in  addition  to 
vertex  constraints  to  generate  interaction  forces  [9,  10]. 

A  smaller  subclass  of  objects  in  our  environment  are  better  classified  as  deformable  me¬ 
dia.  Examples  include  rubber,  foam,  and  biological  tissue.  Deformable  objects  are  compu¬ 
tationally  more  complex  to  simulate  due  to  the  dynamic  behavior  they  exhibit  in  their  local 
reference  frames.  Haptic  interaction  is  simplified  in  rigid  object  simulations  because,  apart 
from  rigid  body  dynamics,  items  1  and  3  above  are  decoupled.  In  other  words,  assuming  all 
objects  are  grounded  in  the  environment,  interaction  forces  with  a  grounded  rigid  object  do 
not  affect  the  state  of  that  object.  Simulation  of  deformable  objects,  conversely,  requires  tight 
computational  coupling  of  items  1  and  3  in  order  to  display  accurate  forces  to  the  user.  The 
deformation  force  and  geometry  of  a  model  is  determined  by  a  combination  of  interaction 
modeling  and  calculation  of  internal  reaction  forces. 

This  thesis  describes  a  prototype  virtual  environment  system,  IDMS  (Interactive  De¬ 
formable  Media  Simulator j,  for  haptically  and  visually  interacting  with  deformable  objects 
in  real-time.  A  discrete  simulation  model  is  used  to  represent  media  by  a  network  of  energy 
storage  and  dissipative  elements  connected  to  point  masses,  or  atoms.  Volumetric  properties 
are  achieved  by  (1)  using  a  two-layer  thick  slab  filled  with  tetrahedral  elements  to  repres¬ 
ent  the  surface  of  the  object  and  (2)  grounding  atoms  on  the  external  surface  of  the  slab  to 
their  respective  equilibrium  positions.  Real-time  interaction  is  achieved  by  assuming  local 
deformation  near  the  point  of  interaction;  computation  power  is  conserved  by  only  simulating 
those  tetrahedral  elements  that  lie  within  a  bounding  sphere  centered  at,  the  interaction  point. 
Models  are  stored  using  pig  and  f am  file  formats  (Appendix  B)  to  visually  and  haptically 
describe  object  properties,  respectively.  Generation  of  models  is  achieved  by  modifying  pre¬ 
existing  models  through  either  programmed  or  manually  controlled  forces.  The  following 
section  discusses  the  motivation  behind  developing  IDMS. 


1.2  Motivation 

Integrated  haptic  and  visual  simulation  of  deformable  media  has  potential  applications  in  a 
broad  range  of  fields.  Applications  in  two  primary  fields  are  briefly  addressed  below. 

•  Medicine 

Virtual  environments  will  play  an  important  role  as  high-performance  training  tools  in 
medicine.  Medical  students  will  be  able  to  learn  about  the  human  anatomy  interact¬ 
ively  by  probing  sophisticated  models  ad  infinitum,  without  accessing  human  cadavers 
or  invading  the  privacy  of  live  human  models.  By  equipping  a  haptic  interface  with 
an  application  specific  tool  such  as  a  scalpel,  students  will  be  able  to  practice  surgical 
procedures,  which  require  a  large  degree  of  hand-eye  coordination,  on  virtual  mod¬ 
els.  Modeling  interaction  with  biological  tissue  in  these  procedures  requires  real-time 
simulation  of  deformable  media, 

*  Computer-Aided  Design 

Current  state-of-the-art  computer-aided  design  (CAD)  tools  lack  the  ability  to  truly 
interact  with  three-dimensional  objects.  Three-dimensional  user  it.  ut  is  transmitted 
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through  two-dimensional  projections  of  vario.  iews  and  a  non-force-reflecting  stylus. 
The  ability  to  both  explore  and  manipula*  h  three-dimensional  force- reflecting 
haptic  interfaces  will  allows  designers  to  ut:.«.  -h'Mr  creativity  much  more  effectively. 
Engineers  could  easily  mold  virtual  clay  to  construct  prototypes  and  proof  their  ideas 
before  deciding  on  a  design.  The  ability  to  interact  with  tangible  three-dimensional 
objects  will  give  engineers  a  heightened  sense  of  confidence  before  spending  precious 
money  on  tooling  and  manufacturing.  The  existing  CAD  primitives  of  cutting,  copying, 
and  pasting  combined  with  the  haptic  channel  of  interaction  will  provide  engineers  with 
powerful  creative  tools. 


3  Organization 

•  Chapter  2  discusses  the  domain  of  deformable  media  that  is  addressed  by  this  thesis. 
Requirements  for  a  real-time  interactive  deformable  media  simulator  are  described. 
Lastly,  a  review  of  literature  related  to  deformable  object  simulation  is  provided. 

•  Chapter  3  describes  and  justifies  the  finite  element  modeling  approach  used  to  cap¬ 
ture  the  significant  physics  of  deformable  media.  Several  features  of  the  approach  are 
described  in  correlation  with  the  requirements  of  Chapter  1. 

•  Chapter  4  describes  the  mathematical  framework  used  for  updating  the  state  of  the 
model  dynamics.  Stability  is  discussed  in  terms  of  the  model  physical  parameters  and 
the  simulation  time  step. 

•  Chapter  5  discusses  the  hardware  and  software  architecture  that  comprise  the  imple¬ 
mentation  of  IDMS.  The  overall  system  loop  that  concurrently  executes  haptic,  envir¬ 
onment,  and  visual  processing  is  also  described.  Algorithms  for  generating  interaction 
force  and  simulating  local  dynamics  are  presented.  Lastly,  techniques  for  automatically 
and  manually  generating  models  are  discussed. 

•  Chapter  6  presents  results  from  fundamental  haptic  interactions  with  IDMS.  Visual 
and  haptic  information  is  presented  for  each  interaction  to  illustrate  the  versatility  and 
functionality  inherent  to  IDMS. 

•  Chapter  7  presents  a  summary  of  the  features  of  IDMS  and  offers  conclusions  for  further 
research  that  will  enhance  the  performance  and  functionality  of  IDMS. 


Chapter  2 

Problem  Domain 


2.1  Scope 

Within  the  perceptible  range  of  the  human  proprioceptive  system,  deformability  qualitatively 
encompasses  a  vast  spectrum  of  objects  ranging  from  highly  compliant  materials  such  as  soft 
foam  to  relatively  stiff  media  such  as  highly  viscous  gels.  Somewhere  along  this  deformab¬ 
ility  spectrum  lies  a  band  of  objects  that  is  meaningful  and  feasible  for  real-time  dynamic 
simulation  in  a  virtual  environment.  As  described  in  the  Introduction,  a  major  application 
area  of  this  research  is  surgery  simulation,  which  requires  realistic  modeling  and  simulation 
of  biological  media.  Without  loss  of  generality,  biological  media  provide  a  meaningful  and 
useful  context  for  addressing  the  issues  inherent  in  designing  an  interactive  deformable  me¬ 
dia  simulation.  A  broader,  perhaps  unrelated,  range  of  deformable  objects  can  be  simulated 
using  this  context  as  a  coherent  perspective.  The  following  assumptions  are  made  about  the 
material  properties  of  biological  media  to  assist  in  the  motivation  of  design  requirements  for 
such  a  system  and  incorporation  into  a  virtual  environment  setting. 


1.  Local  influence 

Many  interactive  medical  procedures  such  as  suturing,  incising,  palpating,  dissecting, 
and  retraction  primarily  have  local  yet  complex  influence  on  the  biological  object. 
The  forces  induced  by  these  interactions,  however,  do  not  substantially  propagate  to 
affect  the  global  state  of  the  object.  This  result  is  fundamentally  due  to  the  highly 
viscoelastic,  overdamped  behavior  of  tissue  and  the  high  mechanical  coupling  with  the 
skeleton  through  connective  tissue  in  the  surrounding  environment. 

2.  Nonhomogeneity 

Biological  media  are  characterized  by  a  non  homogeneous  composition  of  materials  with 
potentially  discontinuous  variations  in  mechanical  impedances.  Some  forms  of  tumor¬ 
ous  tissue,  for  example,  exhibit  much  stiffer  mechanical  properties  than  healthy  tissue. 
Underlying  bone  is  perhaps  better  characterized  by  a  rigid  representation  than  a  de¬ 
formable  one.  Furthermore,  surface  properties  such  as  friction  may  vary  considerably 
depending  on  the  tissue  type. 
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2.2  Requirements 

This  section  focuses  on  the  design  requirements  of  an  interactive  deformable  media  simulation 
environment  given  the  context  described  above. 

2.2.1  Real-time  interactivity 

Real-time  response  in  a  virtual  environment  is  derived  by  the  bounds  on  perceptual  immersion 
defined  by  the  human  sensorimotor  system.  Each  sensorimotor  modality  of  the  human  has 
information  refresh  rates  that  need  to  be  satisfied  or  else  the  perceptual  illusion  of  reality  will 
not  be  conveyed.  The  lower  bound  for  visual  modality  requires  a  throughput  rate  greater 
than  8  to  10  frames  per  second  and  a  maximum  delay  of  0.1  sec  [5,  p.250].  These  values  are 
primarily  for  static  environments  with  slowly  moving  objects.  Environments  which  contain 
higher  frequency  motion  require  frame  update  rates  on  the  order  of  60  Hz  to  achieve  visual 
realism.  The  haptic  modality  requires  considerably  faster  update  rates  to  establish  sufficient 
performance.  Specifically,  the  bandwidth  of  tactile  and  force  feedback  within  the  virtual 
environment  should  exceed  1  kHz  [5,  p.  184] . 

2.2.2  Passivity 

An  important  but  often  overlooked  property  desirable  of  an  interactive  simulation  is  passiv¬ 
ity  of  work  interaction.  Passivity  implies  that  no  net  positive  work  transfer  occurs  from 
hardware  or  software  elements  that  lie  between  the  simulation  and  the  human  during  an  inter¬ 
action.  A  passive  object  in  the  virtual  environment  should  not  appear  active  to  the  human. 
The  environment  dynamics  may  or  may  not  be  passive  depending  on  the  nature  of  the  en¬ 
vironment;  an  example  of  an  active  environment  is  muscle  tissue  in  which  energy  is  being 
produced  internal  to  the  environment.  Although  strict  passivity  is  perhaps  too  conservative 
a  requirement  [2,  3]  for  haptic  interaction,  it  is  a  usefui  robustness  criteria  and  measure  of 
haptic  display  performance. 

2.2.3  Nonhomogeneity 

As  exemplified  above  in  2.1  for  the  case  of  biological  media,  non  homogeneous  material  proper¬ 
ties  such  as  density,  stiffness,  and  viscosity  need  to  be  sufficiently  captured  by  the  modeling 
representation  in  order  to  accurately  convey  changes  in  mechanical  impedance  felt  by  the 
haptic  interface.  Humans  rely  on  the  perceived  continuity  of  change  in  force  to  discriminate 
between  internal  boundaries  of  different  media.  For  example,  doctors  learn  to  detect  tumors 
by  palpating  and  stroking  tissue,  relying  heavily  on  perceptual  changes  in  stiffness  to  identify 
their  presence. 

2.2.4  Reformability 

As  mentioned  above  in  the  context  of  biological  media,  surgical  procedures  involve  cutting, 
removing,  and  joining.  Algorithmically,  the  simulation  should  provide  functionality  for  per¬ 
forming  such  operations  on  the  virtual  model.  More  importantly,  the  model  representation 
should  be  flexible  enough  to  allow  for  such  real-time  alteration  of  object  characteristics  using 
the  haptic  and  visual  channels  provided. 


2.3.  PREVIOUS  APPROACHES 
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2.2.5  Data  file  format 

A  file  format  for  coherently  storing,  reading,  and  writing  model  data  by  the  simulation  is  an 
important  design  requirement.  A  predefined  standard  for  storing  physical  parameters  and 
geometry  allows  for  a  virtually  infinite  set  of  models  to  be  defined  easily.  The  use  of  a  file 
format  also  allows  for  creation  of  models  by  other  software  tools  and  conversion  from  other 
model  definition  protocols  used  by  other  simulations.  Libraries  of  models  can  be  created 
to  represent  systems  that  are  better  represented  as  a  hierarchical  collection  of  independent 
objects,  an  obvious  example  being  the  human  body. 

2.2.6  Haptic  scanning 

In  the  context  of  visual  images,  scanning  converts  the  real  visual  properties  of  an  image  into 
a  representable  data  format  for  later  processing.  Analogously,  the  notion  of  haptic  scanning 
is  introduced.  A  real  object  would  be  haptically  scanned  for  geometric,  mechanical,  and 
material  properties  to  produce  an  artificial  copy  that  would  serve  as  a  haptic  representation  for 
that  object.  This  process  would  allow  virtual  models  to  be  easily  generated  from  existing  real 
objects.  Although  the  mechanics  of  haptic  scanning  are  perhaps  not  appropriately  addressed 
by  the  simulation  itself,  the  modeling  approach  used  in  representing  objects  for  the  simulation 
should  be  rich  enough  and  amenable  to  incorporation  of  data  from  haptic  scanning. 


2.3  Previous  approaches 

This  section  describes  previous  research  relevant  to  deformable  object  simulation. 

Barr  [1]  describes  an  approach  that  permits  global  and  local  deformations  of  solid  geomet¬ 
ries.  Tangent  and  normal  vectors  of  the  deformed  model  geometry  are  achieved  by  applying 
a  transformation  matrix  to  the  tangent  and  normal  vectors  of  the  undeformed  geometry.  A 
series  of  deformation  operations  (including  tapering,  twisting,  bending,  and  scaling)  can  be 
performed  on  undeformed  solid  primitives  by  concatenating  transformation  matrices  in  a 
hierarchical  fashion.  These  methods,  however,  are  not  based  on  underlying  physical  models 
and  hence  do  not  model  nonhomogeneous  physical  behavior  or  allow  for  generation  of  haptic 
interaction  forces. 

Terzopoulos  [23]  formulates  a  method  for  modeling  deformable  objects  using  elasticity 
theory.  Partial  differential  equations  are  derived  using  classical  mechanics  and  solved  using 
finite  element  modeling  techniques.  External  forces  are  added  to  represent  gravity,  fluid 
forces,  and  collisions  with  rigid  objects.  A  similar  framework  is  used  by  Ter.  >poulos  [24] 
to  address  the  inelastic  deformation  behaviors  of  viscoelasticity,  plasticity,  and  fracture.  The 
resulting  simulation  of  object  dynamics  are  realistic  and  accurately  represent  natural  phys¬ 
ical  behavior.  The  finite  element  calculations  used  in  each  of  these  methods,  however,  are 
prohibitive  for  real-time  simulation  and  thus  do  not  satisfy  the  interactivity  requirement. 

Pentland  [16,  17]  presents  a  visually  real-time  (15  Hz)  solid  modeling  system  that  allows 
users  to  mold  “clay”  by  applying  external  virtual  forces.  A  family  of  superquadrics  are  used 
as  the  basic  volume  primitives  for  describing  the  object  geometry.  The  finite  element  method 
is  utilized  to  derive  the  matrix  differential  equations  describing  the  dynamic  behavior  of  each 
object.  Computational  expense  is  minimized  by  using  a  modal  method  to  only  simulate  the 
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precomputed  low-frequency  modes  of  the  system.  The  approach,  however,  does  not  address 
generation  of  haptic  interaction  forces  nor  real-time  simulation  at  haptic  refresh  rates  («  l 
kHz).  Furthermore,  interactive  reformabiiity  does  not  fit  within  the  modal-based  framework 
since  modification  of  object  structural  properties  would  require  expensive  recomputation  of 
the  modal  vectors. 

Pieper  [18]  describes  a  system  for  animating  human  facial  tissue  using  a  finite  element 
approach  to  globally  model  tissue  behavior.  The  structural  and  geometric  topology  is  rep¬ 
resented  using  a  discrete  simulation  model,  a  network  of  discrete  masses  and  mechanical 
elements  stored  in  graph  form.  This  representation,  similar  to  the  one  used  in  this  thesis, 
is  appropriate  for  describing  non  homogeneous  physical  parameters  and  nonlinear  mechanical 
relationships.  The  graph  method  is  also  highly  amenable  to  reformabiiity,  facilitating  addi¬ 
tion  and  removal  of  nodes  and  links  during  run-time.  Pieper’s  approach  for  modeling  facial 
tissue,  however,  is  not  appropriate  for  modeling  volumetric  behavior  of  deformable  objects. 
Furthermore,  haptic  interaction  issues  are  not  addressed.  Pieper  [19]  also  discusses  a  system 
for  simulating  plastic  surgery  that  is  based  on  finite  element  modeling.  A  global  stiffness 
matrix  is  utilized  for  solving  the  differential  equations.  This  method,  however,  is  aimed  at 
task-level  analysis  and  similarly  does  not  address  display  of  information  through  the  haptic 
channel. 

Cover  [4]  presents  a  methodology  for  interactive  deformation  of  surfaces  by  using  the 
concept  of  active  contours,  essentially  energy-minimizing  splines.  The  surface  is  discretized 
as  a  mesh  of  points  and  spring-like  connections,  each  point  containing  information  about 
its  direct  neighbors  and  the  external  force  being  applied  to  the  point.  A  “home”  force  is 
also  applied  that  acts  to  restore  each  point  to  its  equilibrium  position.  A  gradient  descent 
approach  is  used  to  update  the  point  positions  in  the  mesh  in  an  energy-minimizing  manner. 
The  internal  energy  of  the  contour,  however,  is  globally  controlled  by  only  two  parameters 
that  affect  elasticity  and  flexibility.  Hence,  local  variation  in  physical  characteristics  are 
not  captured  by  the  active  contour.  Furthermore,  it  is  not  clear  whether  this  approach  is 
amenable  to  easy  incorporation  of  haptic  scan  data.  Lastly,  real-time  interactivity  is  not 
addressed  in  the  framework  of  haptics. 

Each  of  these  methods  are  missing  fundamental  features  that  are  necessary  to  satisfy 
the  requirements  for  true  simultaneous  haptic  and  visual  interaction  with  deformable  media. 
The  following  chapter  discusses  the  modeling  approach  used  in  IDMS  and  appeals  to  the 
requirements  listed  above. 


Chapter  3 

Physical  Modeling 


Modeling  of  deformable  objects  is  indeed  a  nontrivial  task.  Unlike  rigid  objects,  deform¬ 
able  media  have  significant  internal  dynamics  and  thus  a  state-based  approach  must  be  used 
to  solve  for  interaction  forces.  A  physically-based  approach  grounded  in  applied  mechanics 
ensures  that  the  model  will  capture  the  real  world  idiosyncrasies  of  deformable  media.  A 
molecular  level  model,  however,  is  clearly  not  necessary  for  a  virtual  environment  setting  in 
which  perceptual  and  cognitive  abilities  limit  the  range  of  information  that  can  sensed.  Thus, 
a  viable  model  must  capture  sufficient  properties  to  prove  perceptually  realistic  while  main¬ 
taining  enough  simplicity  to  allow  for  real-time  simulation.  The  following  chapter  describes 
the  fundamentals  behind  the  modeling  approach  used  in  IDMS  and  the  features  that  make 
the  approach  highly  amenable  for  haptic  interaction. 


3.1  Approach 

The  modeling  approach  used  in  IDMS  utilizes  a  combination  of  finite  element  modeling 
(FEM)  theory  and  surface  encapsulation  of  volumetric  properties  to  convincingly  convey 
the  feeling  of  deformation  to  the  user.  In  mathematics,  a  modal  decomposition  separates 
a  dynamic  system  into  a  linear  superposition  of  normal  eigenvectors.  Carrying  this  modal 
decomposition  metaphor  over  to  physical  modeling,  the  haptic  model  only  needs  to  represent 
the  most  significant  modes  of  deformation  to  the  user,  The  model  is  composed  of  a  two- 
dimensional  layer  of  finite  tetrahedral  structural  elements,  networked  together  to  form  the 
surface  of  the  deformable  model  (see  Figure  3-1).  Tetrahedral  elements  (dashed  line)  are  used 
because  they  are  structurally  stable  and  have  low  mass  density,  requiring  minimal  discrete 
elements  per  unit  volume  to  represent  them.  The  encapsulation  of  volumetric  effects  is 
achieved  by  adding  a  restoring  force  between  the  top  surface  (bold  line)  of  the  tetrahedral 
mesh  and  the  equilibrium  position  of  the  top  surface.  The  user  only  interacts  with  the 
top  surface  of  the  mesh,  both  visually  and  haptically.  The  notion  of  a  haptic  window  is 
introduced  to  allow  simulation  of  object  dynamics  local  to  the  interaction  point.  Only  media 
that  falls  within  this  geometrically  shaped  window  is  simulated  in  order  to  generate  real-time 
response.  Combined,  these  features  serve  to  haptically  and  visually  display  the  significant 
modes  of  deformation  to  the  user. 

A  discrete  simulation  model  (DSM)  is  chosen  to  represent  the  surface  mesh  described 
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Figure  3-1:  A  fragment  of  media  represented  by  a  two-dimensional  layer  of  tetrahedral 
elements  (dashed  line).  Volumetric  effects  are  simulated  by  applying  a  restoring  force  between 
the  top  surface  (bold  line)  and  the  equilibrium  position  of  the  top  surface. 


above.  A  DSM  is  a  discretized  network  of  lumped  parameter  elements  and  atoms  that  ap¬ 
proximates  the  physical  behavior  of  a  continuum  media.  In  the  present  context  of  deformable 
objects,  atoms  are  point  masses  connected  to  neighboring  atoms  through  constitutive  mech¬ 
anical  elements.  Each  tetrahedral  element  is  represented  by  four  link  elements  and  four 
bounding  atoms,  shared  with  neighboring  tetrahedra.  The  DSM  is  stored  in  graph  form  to 
facilitate  alteration  of  the  model  topology.  Simulation  of  the  DSM  state  is  accomplished  by 
a  multi-step  algorithm  that  is  evaluated  within  the  topology  of  the  model.  During  each  time 
step,  forces  are  summed  for  each  mobile  atom  based  on  external  forces  and  internal  forces 
from  neighboring  elements.  The  two  state  variables  of  position  and  velocity  for  each  atom 
are  then  numerically  integrated  resulting  in  the  DSM  state  for  the  next  time  step. 

The  restoring  force  between  the  top  layer  of  the  mesh  and  the  equilibrium  position  is 
achieved  within  the  framework  of  the  DSM.  Essentially,  a  massless  immobile  home  atom 
is  added  at  the  equilibrium  position  of  each  atom  located  on  the  surface  of  the  mesh.  A 
connecting  home  element  is  added  between  the  surface  atom  and  its  respective  home  atom. 
As  the  virtual  mesh  is  deformed  interactively,  the  superposition  of  forces  produced  by  home 
elements  in  tension  compactly  simulates  the  volumetric  compression  of  the  continuum  media. 
Figure  3-2  displays  the  equilibrium  and  deformation  states  of  a  media  fragment,  explicitly 
showing  the  home  elements  and  atoms  for  the  surface  atoms  that  are  significantly  deformed1. 

The  following  section  describes  the  features  of  the  modeling  approach  mentioned  above  in 
light  of  the  requirements  of  an  interactive  deformable  object  simulation  described  in  Chapter 
2. 


'Note  that  during  a  normal  IDMS  session,  only  surface  elements  and  surface  atoms  are  visually  rendered. 
Home  elements,  home  atoms,  and  interior  atoms  and  elements  are  visually  hidden. 
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Figure  3-2:  A  fragment  of  dsm  media  in  equilibrium  is  shown  on  the  left.  A  sample  deform¬ 
ation  and  the  home  atoms  and  elements  are  explicitly  shown  at  right. 


3.2  Features 

3.2.1  Surface  representation 

Assuming  small  deformations,  haptic  interaction  can  be  viewed  as  primarily  a  surface  phe¬ 
nomena.  Although  the  objects  being  modeled  are  clearly  three-dimensional  volumes,  the 
interactions  being  modeled  can  be  approximated  as  occuring  ner,r  the  object  surface.  As  dis¬ 
cussed  below,  a  surface  representation  of  haptic  properties  is  spatially  and  computationally 
efficient  compared  to  volumetric  models  of  object  physics. 


Spatial  properties 

From  a  spatial  perspective,  a  surface  representation  >s  highly  desirable.  The  amount  of 
geometric  and  haptic  data  required  to  describe  the  object  scales  in  proportion  to  the  surface 
area  instead  of  the  object  volume.  Furthermore,  the  process  of  haptically  scanning  in  real 
object  properties  will  likely  be  implemented  by  measuring  surface  impedance  and  material 
properties  through  the  use  of  an  instrumented  force-measurement  device.  Mathematically 
speaking,  this  procedure  can  be  described  as  projecting  multivariable  data  captured  from  a 
measurement  input  space  onto  the  geometric  output  space.  Given  that  measurement  data  will 
spatially  cluster  about  the  surface  of  the  object,  the  use  of  a  surface  model  for  representing 
haptic  properties  facilitates  the  data  projection  anu  registration  process.  Conversely,  use  of  a 
volumetric  model  would  imply  projection  of  the  measurement  space  onto  a  higher  dimensional 
space,  requiring  the  use  of  interpolation  or  other  data-insertion  techniques  to  fill  the  output 
space. 

The  home  element  described  above  is  a  discretized  encapsulation  of  the  local  surface 
impedance  and  volumetric  properties  in  the  modeled  continuum  object.  In  the  current  gener¬ 
ation  of  IDMS,  each  home  element  is  a  second  order  representation  of  the  surface  impedance, 
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storing  elastic  and  viscous  properties.  The  spatial  density  of  home  elements  is  governed  by 
the  sampling  size  of  tetrahedral  elements  used.  Borrowing  principles  from  discrete-time  sig¬ 
nal  processing  [15],  a  larger  resolution  of  tetrahedra  will  capture  higher  spatial  frequency 
con^nt  of  the  continuum  media.  A  larger  resolution,  however,  increases  data  storage  and 
imposes  a  greater  computational  load  on  the  haptic  rendering  algorithm  for  displaying  the 
data. 


Computational  properties 

A  surface  representation  allows  for  computationally  tractable  rendering  of  virtual  objects. 
Real-time  simulation  of  objects  using  volumetric  finite  elements  throughout  the  object  are  still 
intractable  even  on  supercomputers.  A  surface  model  reduces  the  computational  complexity 
from  requiring  0(n3)  operations  for  a  volume  representation  to  only  0(n2)  operations  for  a 
surface  representation.  Given  this  reduction,  desktop  workstations  can  suffice  as  real-time 
simulation  engines. 


3.2.2  Local  simulation 

Current  haptic.,  technology  primarily  involves  environment  interaction  with  fingers  or  tools 
that  have  localized  mechanical  effect.  Just  as  a  graphics  simulation  only  renders  those  objects 
that  fall  within  the  viewing  window  of  the  monitor,  a  haptics  simulation  need  only  render 
those  efiects  that  are  spatially  local  to  the  interaction  point  to  minimize  computation  effort. 
This  haptic  “window”  is  not  as  spatially  rigid  as  the  rectangular  viewing  space  of  a  tradi¬ 
tional  graphics  disolay  because  physical  elements  throughout  the  model  contribute  some  work 
transfer  to  the  user.  Thus,  the  haptic  window  must  selectively  include  enough  elements  to 
the  point  of  diminishing  marginal  return  for  computational  cost  versus  perceptual  realism. 

A  DSM  facilitates  local  simulation  of  dynamics.  As  explained  in  more  detail  in  Section 
5.5.2,  local  simulation  is  achieved  by  only  simulating  media  that  lies  within  a  sphere  centered 
at  the  interaction  point;  the  rest  of  the  model  is  immobilized  and  hence  not  unnecessarily  sim¬ 
ulated  during  each  time  step.  The  size  of  this  sphere  is  empirically  chosen  such  that  stroking 
along  the  surface  of  the  virtual  object  is  haptically  smooth.  Although  not  implemented  in  .he 
current  generation  of  IDMS,  a  slower  background  process  could  be  added  to  update  global 
dynamics.  This  parallelism  of  different  time  scale  processes  is  amenable  to  the  concurrent 
run-time  model  used  in  the  software  implementation  of  IDMS  (Section  5.2). 


3.2.3  Nonhomogene  ity 

Non  homogeneity  of  model  pnysical  characteristics  are  compactly  represented  in  the  DSM 
framework.  Variations  in  density  can  be  achieved  varying  atom  inass  values  appropriately. 
Nonhomogeneous  stiffness  and  viscosity  is  similarly  achieved  by  varying  the  physical  para¬ 
meters  of  DSM  elements.  Furthermore,  the  DSM  framework  conveniently  allows  for  regions  of 
higher  resolution  meshing  of  tetrahedral  elements,  especially  where  surface  properties  contain 
higher  spatial  frequency. 
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3.2.4  Nonlinearity 

In  addition  to  nonhomogeneity,  many  materials  exhibit  nonlinear  mechanical  behavior.  Using 
a  solid  mechanics  model,  the  regime  below  the  yield  stress  of  a  material  can  be  represented 
by  linear  relations;  above  this  point,  the  material  becomes  plastic  and  nonlinear  stress-strain 
behavior  results.  Given  the  graph  form  (Section  3.3)  used  to  store  the  DSM,  each  element 
may  contain  a  nonlinear  constitutive  relation  F(/,/)  to  produce  an  internal  force.  Additional 
state  variables  may  be  added  per  element  as  necessary  for  other  force  generating  relations. 
For  simplicity  of  computation  and  modeling,  however,  all  the  models  used  in  this  thesis  use 
linear  relations. 

3.2.5  Boundary  conditions 

Boundary  conditions  can  be  easily  applied  by  immobilizing  atoms.  Each  atom  data  structure 
contains  a  boolean  specifying  the  boundary  condition,  fixed  or  free,  of  the  atom  (see  Section 
3.3).  The  DSM  simulation  algorithm  saves  computation  power  by  only  updating  the  state  of 
those  atoms  that  have  a  iree  boundary  condition.  Also,  element  forces  are  generated  only  for 
elements  that  are  bounded  by  at  least  one  mobile  atom. 

3.2.6  Reformability 

The  graph-based  method  (Section  3.3)  and  data  structures  used  to  represent  the  DSM  greatly 
facilitate  appending  and  removing  atoms  and  elements  from  the  model.  Different  interaction 
procedures  can  be  implemented  by  producing  different  effects  on  the  topology  of  the  graph. 
For  example,  removal  of  material  can  be  achieved  by  deleting  atoms  and  elements  from  the 
graph.  Slicing,  a  mass-conserving  procedure,  can  be  accomplished  by  removing  elements 
along  the  cutting  path.  Gluing  and  joining  multiple  models  together  is  possible  by  adding 
elements  between  existing  atoms  of  the  models. 

3.2.7  Parallel  processing 

The  DSM  approach  scales  well  with  increased  p  ocessing  power,  both  in  single  and  multi¬ 
processor  computer  architectures.  More  processing  power  translates  into  a  greater  number  of 
atoms  and  elements  that  can  be  simulated  at  required  haptic  servo  rates,  allowing  rendering 
of  more  complex  models.  In  a  multi-processing  system,  each  processor  can  be  assigned  a 
region  of  the  model  to  simulate.  Run  time  model  data  would  be  stored  in  central,  shared 
memory  to  facilitate  data  coherency  and  access  between  processors. 

3.2.8  Data  file  format 

The  lem  and  pig2  file  formats  described  in  Appendix  B  are  the  off-line  storage  protocols  for 
representation  of  model  data  used  in  IDMS.  All  initial  conditions,  physical  parameters,  and 
topoiogy  information  used  in  constructing  the  data  structures  and  connectivity  are  completely 
contained  in  these  files. 


2The  pig  file  format  is  a  subset  of  the  more  descriptive  Wavefront  Technologies  ob j  file  format,  which 
will  be  integrated  into  later  iterations  of  IDMS 
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3.3  Data  representation 

Storing  the  DSM  data  in  graph  form  allows  for  flexibility  and  efficiency  in  computation.  Each 
node  in  the  graph  represents  an  atom  and  each  link  represents  an  element.  In  conventional 
FEM  techniques,  matrices  are  used  to  store  state  and  material  property  as  the  simulation 
progresses.  Often  these  matrices  are  sparsely  filled,  thus  occupying  an  unnecessary  amount 
of  memory.  For  a  model  with  N  atoms  and  two  state  variables  per  atom,  use  of  matrix 
storage  amounts  to  0(4N2)  storage.  Graph  storage,  conversely,  grows  linearly  with  N  and 
is  only  O(N).  Graph  storage  is  highly  amenable  to  the  design  requirement  of  reformability. 
As  material  is  removed,  added,  or  joined  during  the  simulation,  nodes  and  links  in  the  graph 
can  be  correspondingly  manipulated.  Matrix-based  methods  are  essentially  limited  to  static 
topologies,  because  appending  and  deletion  are  algorithmically  difficult  and  time-consuming. 

In  addition  to  the  spatial  advantages  of  linear  storage  and  malleability,  graph  methods 
are  time  efficient  relative  to  matrix  methods.  Complex  computationally-intensive  numerical 
techniques  are  necessary  to  invert  matrices.  Assuming  the  best  case  scenario  for  matrix 
methods,  a  matrix  multiplied  by  a  state  vector  must  occur  during  every  simulation  loop. 
This  operation  entails  0(8N3)  operations.  Conversely,  solving  for  the  state  using  graph 
methods  only  requires  0(N  +  L)  operations,  where  L  is  the  number  of  elements  in  the  model. 
The  following  sections  describe  the  data  structures  that  comprise  a  DSM, 

3.3.1  Atom 

The  data  structure,  A ,  for  the  atom  is  shown  in  Table  3.13.  The  variable  num  contains  the 
atom’s  number  in  the  list  of  atoms  for  the  parent  model,  M.  A  list  of  neighboring  element 
structure  addresses  is  maintained  to  facilitate  in  calculation  of  external  forces  on  the  atom. 
The  surface  condition  sc  holds  a  boolean  determining  whether  the  atom  is  located  on  the 
exterior  of  the  model  (external),  hence  visually  and  haptically  sensable,  or  on  the  interior 
(internal).  The  boundary  condition  be  is  also  a  boolean  determining  whether  a  position 
constraint  is  being  imposed  (fixed)  or  not  (fre<  "inally,  the  spatial  condition  spe  specifies 
if  the  atom  is  inside  or  outside  the  haptic  window'. 

3.3.2  Element 

The  element  data  structure  is  shown  in  Table  3.2.  The  num  contains  the  element’s  number 
in  the  li3t  of  elements  under  the  parent  model,  M.  The  physical  parameters  for  the  element 
are  stored  in  variables  k ,  b,  and  leq.  Structure  addresses  &*4°  and  &.41  of  the  Cement's 
two  bounding  atoms  allow  the  state  variables  l  and  l  to  be  calculated.  The  state  variables 
combined  with  the  physical  parameters  are  used  to  evaluate  the  internal  element  force  f 
during  each  simulation  loop. 

3.3.3  Model 

The  model  data  structure,  M,  is  the  root  node  of  the  DSM  graph  and  stores  individual  lists 
of  the  N  atoms  and  L  elements  that  comprise  it  (Table  3.3).  As  a  preprocessing  step,  a  list 


3  ANSI  C  notation  is  used  where  appropriate  for  addressing. 
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>1 

num 

atom  number 

m 

mass 

P 

absolute  position 

absolute  equilibrium  position 

Xl 

differential  position 

X3 

differential  velocity 

Xl 

time  derivative  of  xi 

x3 

time  derivative  of  X2 

f 

net  force  applied  on  atom 

. k£k~l 

list  of  neighboring  elements 

be 

boundary  condition  (fixed  or  free) 

sc 

surface  condition  (external  or  internal) 

spe 

spatial  condition  (local  or  non-local) 

Table  3.1:  Data  structure  for  atom. 


£ 

num 

element  number 

k 

stiffness 

b 

viscosity 

l 

length 

i 

time  derivative  of  length 

leq 

equilibrium  length 

f 

net  internal  force 

&A°,kAl 

list  of  neighboring  atoms 

Table  3.2:  Data  structure  for  element. 
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M 

name 

model  name 

kA°,kA\. 

list  of  model  atoms 

kA°,kA\.. 

. ,kANS -1 

list  of  model  surface  atoms 

k£°,k£l, 

list  of  model  elements 

Table  3.3:  Data  structure  for  model. 

of  the  N S  atoms  which  are  on  the  surface  of  the  model  is  also  included  in  the  model  data 
structure.  This  list  of  surface  atoms  facilitates  evaluation  of  collision  detection  algorithms  by 
reducing  the  size  of  the  search  space;  atoms  that  are  beneath  the  surface  need  not  be  searched 
for  potential  interaction  with  the  user. 


Chapter  4 

Mathematical  Framework 


4.1  Simulation 

A  state-space  approach  is  used  to  analyze  and  simulate  the  model  dynamics.  Briefly,  in 
a  state-space  approach,  the  mathematical  model  is  represented  by  a  series  of  first  order 
differential  equations.  The  order  of  a  dynamic  system  is  determined  by  the  number  of  initial 
conditions  that  need  to  be  specified.  In  the  DSM  approach,  there  are  two  state  variable 
vectors,  xi  and  xj,  for  each  of  the  N  atoms  in  the  model,  resulting  in  at  least  a  6 N  order 
system  for  the  entire  model.  6 N  is  a  lower  bound  on  the  order  of  the  system  because  the 
constitutive  relations  used  to  calculate  internal  element  forces  may  be  nonlinear.  The  set  of 
differential  equations  that  defines  the  dynamic  process  of  a  model  can  be  written  as 

x  =  /(x,u,f)  (4.1) 

where  x  is  a  6N  dimensional  state  vector  of  the  process.  Although  the  physics  defined  in 
the  element  constitutive  relations  are  time-invariant,  they  have  the  potential  to  be  nonlinear 
functions  of  x.  Thus,  the  lack  of  linear  time  invariance  (LTI)  in  the  model  precludes  use  of 
the  conventional  convolution  integral  approach  for  solving  the  state  of  the  dynamic  system. 

As  will  be  explained  late',  the  actual  run-time  order  of  the  model  is  much  less  because 
only  certain  atoms  are  simulated  each  time  step,  depending  on  the  position  of  the  haptic 
interface.  This  “localizing”  simulation  feature  due  to  use  of  a  haptic  window  is  fundamental 
to  realizing  real-time  deformation  response.  During  the  simulation,  a  multi-step  algorithm 
shown  in  Figure  4-1  is  used  iteratively  to  update  the  state  of  the  model.  Each  block  of  the 
diagram  is  explained  in  detail  below. 

1.  Calculate  external  forces 


f  =  £4<  (4-2) 

The  external  force  on  each  mobile  atom,  *4’,  in  the  DSM  is  calculated,  as  shown  in 
Equation  4.2,  and  stored  in  the  atom’s  data  structure.  This  step  essentially  finds  the 
input  component  /( u)  in  Equation  4.1.  The  external  input  can  be  supplied  by  a  variety 
of  sources.  Field  forces  such  as  gravity  (f‘It  =  m'g)  and  air  resistance  (f’rt  =  -m'x\) 
may  be  applied  to  each  mobile  atom.  Interaction  forces  are  applied  locally  to  those 
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..A 


EVALUATE  TIME 
DERIVATIVE  OF 
EACH  ATOM'S  STATE 
VARIABLES 


Figure  4-1:  Algorithm  for  updating  the  DSM  state. 

surface  atoms  that  come  into  contact  with  the  haptic  interface  (see  Section  5.5.1).  The 
versatility  of  the  DSM  allows  other  external  forces  to  be  easily  added  as  the  environment 
model  becomes  more  complex.  Examples  include  interaction  with  rigid  environment 
walls,  collision  between  multiple  deformable  objects,  md  surface  friction  at  the  contact 
patch. 

2.  Calculate  internal  element  forces 

^nce  all  external  forces  on  mobile  atoms  are  calculated,  internal  element  forces  need 
to  be  evaluated  and  stored  in  each  element’s  data  structure,  S.  This  step  is  spatially 
.solving  the  potentially  nonlinear  component  /(x)  in  the  topology  of  the  model.  The 
constitutive  element  relations,  nonlinear  or  linear,  are  determined  during  the  modeling 
process  for  the  particular  media  being  simulated.  Currently,  each  element  is  composed 
of  a  linear  spring  and  dashpot  in  parallel  and  hence  apply  only  linear  elastic  and  viscous 
forces,  respectively.  The  following  equations  are  used  to  update  each  element’s  state. 

Equation  4.3  finds  the  difference  vector,  s,  between  the  absolute  positions  of  the  two 
bounding  atoms  A1  and  .4°. 

s  =  r-K4*->p  -  f -k4°->p  (4.3) 


» 


i 


» 


•  j 
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The  element’s  length,  /,  for  the  current  time  step  is  found  by  taking  the  magnitude  of 
s,  as  shown  in  Equation  4.4. 


£'->/  =  |s| 


(4.4) 


The  unit  direction  vector,  s,  between  the  element’s  adjoining  atoms  is  consequently 
found  by  Equation  4.5. 


(4.5) 


The  scalar  velocity  of  the  element’s  length,  /,  is  then  found  by  applying  a  first  order, 
backwards  difference  filter  to  the  scalar  length  data  for  that  element  (Equation  4.6). 


r-+/n  =  \(£'-+ln  -  £’->/n_1)/r+  (1  -  A jw;.!  (4.6) 


The  magnitude  of  stiffness  force  present  in  the  element  is  found  by  multiplying  the 
element’s  stiffness  constant,  k,  with  the  difference  between  current  element  length,  /, 
and  element  equilibrium  length,  leq.  The  stiffness  force  vector  is  found  by  scaling  s  with 
the  stiffness  force  magnitude  (Equation  4.7). 

fk  =  r-4fc(f*-+ /  -  r-+/e,)s  (4j) 

The  viscous  force  is  similarly  found  by  multiplying  the  element’s  viscous  constant,  b, 
with  /,  and  consequently  scaling  s  by  this  amount  (Equation  4.8). 

f6  =  £‘-»6(£*->/)s  (4.8) 


Finally,  the  net  force  internal  to  the  element  is  found  by  summing  up  the  vector  force 
contributions  of  the  various  constitutive  elements  (Equation  4.9). 

r-4f  =  ffc+ffc  (4.9) 


3.  Evaluate  time  derivatives  of  state  variables 

The  next  block  evaluates  the  time  derivatives  of  the  state  variables  Xj  and  Xj  for  each 
of  the  mobile  atoms  in  the  model.  Equation  4.10  assigns  the  velocity,  xj,  to  the  time 
derivative  of  position,  Xj. 

xi  =  x2'  (4.10) 

The  acceleration,  x'2,  is  set  equal  to  ^  multiplied  by  the  sum  of  forces  acting  on  A: 
as  shown  in  Equation  4.11.  The  sum  of  forces  is  comprised  of  external  forces  *4'  ->f, 
calculated  in  Step  1,  and  internal  forces  YljZ o  calculated  in  Step  2.  Only  the 

k  elements  that  connect  to  A'  are  used  in  calculating  the  internal  forces.  This  step 
completes  the  vector  differential  equation  defined  in  Equation  4.1. 

1  k- 1 

xy  =  +  £  £J->f)  (4.ii) 

m  “ 


4.  Integrate  state  variables 

Several  numerical  algorithms  exist  for  integrating  ordinary  differential  equations  of  the 
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form 

x  =  /(x,f),  (4.12) 

where  the  function  /  is  known  [20].  These  algorithms  can  be  broadly  grouped  into  the 
following  three  classes. 

(a)  Predictor-corrector 

(b)  Bulirsch-Stoer 

(c)  Runge-Kutta 

The  dynamic  process  of  a  DSM,  however,  obeys  the  differential  equation  shown  in 
Equation  4.1,  which  has  dependence  on  a  causal  input  forcing  vector  u.  Given  that  the 
virtual  environment  simulation  is  progressing  concurrently  with  non-deterministic  user 
interaction,  haptic  forces  can  only  be  calculated  for  the  same  time  interval  as  is  being 
simulated. 

Although  predictor-corrector  (PC)  methods  do  not  require  evaluations  of  /  at  time  steps 
future  to  the  current  one  (and  hence  obey  causality),  PC  methods  are  only  appropriate 
for  very  smooth  equations.  Given  the  potential  for  discontinuous  human  input  through 
the  haptic  interface,  PC  methods  are  not  desirable.  Similar  to  PC  methods,  the  Bulirsch- 
Stoer  (BS)  method  is  only  appropriate  for  differential  equations  containing  smooth 
functions  on  either  side.  Furthermore,  the  BS  method  requires  evaluations  of  /  at  later 
time  steps,  violating  the  causality  criteria. 

Runge-Kutta  methods,  of  the  algorithms  listed  above,  are  the  most  amenable  to  integ¬ 
rating  nonsmooth  data.  An  nth  order  Runge-Kutta  method  makes  use  of  n  function 
evaluations  to  calculate  the  state  of  the  next  time  step.  Although  used  in  the  literature 
primarily  for  pedagogical  purposes,  the  Euler  method  (n  =  1)  satisfies  causality  and 
is  computationally  efficient  since  the  additional  computational  cost  of  re-evaluating  the 
DSM  loop  for  higher  order  Runge-Kutta  methods  is  prohibitive.  Currently,  the  Euler 
method  has  proven  sufficient  for  simulating  a  wide  range  of  material  properties  in  IDMS. 

Equations  4.13  and  4.14  integrate  state  variables  xj  and  Xj,  respectively,  using  Euler’s 
method  and  the  simulation  step  size,  T. 

xi[n+  1]  =  xi[n]  +  xi[n]T  (4.13) 

x2[n-f- 1]  =  X2[n]  +  x3[n]T  (4.14) 

Equation  4.15  updates  the  absolute  position  p  for  each  atom  by  adding  the  differential 
position  Xi  to  the  absolute  equilibrium  position  peq. 

p[n+  1]  =  pe,  +  xi[n+  1]  (4.15) 


4.2  Stability 

The  numerical  stability  of  the  DSM  simulation  is  governed  by  a  combination  of  stability 
of  the  continuous-time  model,  step  size  T,  and  the  discrete-time  integration  method  used. 
Aside  from  guaranteeing  a  stable  response,  addressing  stability  is  important  for  maximizing 
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computational  efficiency.  A  time  step  that  is  chosen  too  targe  for  the  given  physical  parameters 
will  reduce  simulation  speed  unnecessarily,  hence  compromising  haptic  fidelity. 

Given  the  infinite  set  of  complex  three-dimensional  topologies  that  can  be  simulated 
using  IDMS,  it  is  impossible  to  produce  a  stability  criteria  for  all  models. Much  insight  can 
be  derived,  however,  from  analyzing  a  lumped  parameter  model  of  a  media  fragment  and 
using  the  resulting  intuition  in  designing  global  models.  Stability  of  the  models  used  in  this 
thesis  was  determined  by  mathematically  analyzing  a  fragment  of  DSM  media,  creating  a 
global  object  with  this  media,  and  then  empirically  testing  the  object  haptically  for  proper 
feel.  This  procedure  is  repeated  as  necessary  to  converge  on  a  perceptually  convincing,  yet 
stable  model. 

The  system  shown  in  Figure  4-2  is  a  one-dimensional  simplification  of  the  haptic  win¬ 
dow  described  in  Section  3.2.2.  Each  mass  is  connected  by  a  spring-dash  pot  element  to  its 
nearest  neighbors  and  its  own  equilibrium  position,  hence  modeling  the  connecting  and  home 
elements,  respectively.  The  ends  of  the  system  are  mechanically  grounded,  thus  representing 
the  boundary  condition  at  the  haptic  window  border.  This  simple  6th  order  system  is  an 
excellent  vehicle  for  illustrating  the  stability  of  the  discrete-time  window  model  as  a  result  of 
varying  physical  parameters  and  T. 


Figure  4-2:  6th  order  system  model  of  haptic  window. 
The  continuous  time  differential  equation  for  the  plant  is 


x  —  Ax 


(4.16) 


where 
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are  the  system  matrix  A  and  state  vector  x  respectively. 

Using  an  Euler  integration  method,  the  discrete-time  equation  describing  the  state  update 
is 


x[n  +  l]  =  {AT  +  /)x[n] 


(4.18) 
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where  T  is  not  to  be  confused  as  a  matrix.  The  stability  of  the  system  is  governed  by 
the  eigenvalues  of  the  discrete-time  system  matrix,  A,  =  AT  A  /.  The  following  sections 
describe  the  tradeoff  between  performance  and  stability  in  the  z-plane  using  a  root  locus 
approach.  Nominal  unitless  parameter  values  of  the  media  fragment  are  k  =  15.0,  6  =  0.45 
and  m  =  0.002  and  the  time  step  is  T  =  0.001  sec,  resulting  in  an  underdamped  system.  In 
the  following  figures,  ‘x'  marks  the  location  of  the  nominal  pole  positions.  In  each  subsection, 
one  parameter  is  varied  while  holding  the  others  fixed. 

4.2.1  Stiffness 

As  shown  in  Figure  4-3,  increasing  the  element  stiffness  k  decreases  the  margin  of  stability 
in  the  z-plane  for  a  fixed  value  of  T.  Stiffer  systems  have  higher  natural  frequencies  and  thus 
require  smaller  time  steps  for  stable  numerical  simulation.  Nevertheless,  at  T  —  0.001  sec  (1 
kHz  update  rate),  the  range  of  representable  stiffnesses  is  still  greater  than  10:1. 


Real 

Figure  4-3:  Z-plane  root  locus  of  haptic  window:  15  <  k  <  400. 


4.2.2  Mass 

Decreasing  the  atom  mass  m  forces  the  poles  to  the  left  on  the  real  axis  (see  Figure  4-4, 
eventually  pushing  them  outside  the  unit  circle.  From  a  performance  perspective,  however, 
lighter  masses  are  desirable  because  they  reduce  the  amount  of  high  frequency  momentum 
transfer  during  interaction  with  the  surface.  Larger  masses  result  in  the  perception  of  haptic 
“noise”  as  atom  masses  transition  on  and  off  the  master  interaction  point  (see  Section  5.5.1). 
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Figure  4-4:  Z-plane  root  locus  of  haptic  window:  0.001  <  m  <  0.002. 

4.2.3  Viscosity 

In  the  continuous-time  system,  decreasing  the  viscosity  6  moves  the  poles  closer  to  the  jw 
axis  while  increasing  the  viscosity  stabilizes  the  system.  In  the  discrete-time  case,  lowering 
the  viscosity  similarly  forces  the  poles  towards  the  unit  circle,  for  example,  shown  at  the 
value  of  b  —  0.1  in  Figure  4-5.  Larger  values  of  b  increase  the  frequency  content  of  the  plant 
dynamics,  also  tending  to  destabilize  the  system  (6  =  0.9).  For  the  nominal  value  of  b  =  0.45, 
a  viscosity  range  of  approximately  10:1  exists. 

4.2.4  Time  step 

From  a  performance  perspective,  a  minimal  time  step  T  is  desired  in  order  to  increase  the 
servo  rate  of  the  haptic  display,  hence  improving  the  bandwidth  of  force  display.  Furthermore, 
from  a  stability  viewpoint,  increasing  T  shifts  the  poles  to  the  left  on  a  path  leaving  the  unit 
circle,  as  shown  in  Figure  4-6.  A  smaller  time  step,  however,  limits  the  number  of  atoms  and 
elements  that  can  be  simulated  each  time  step,  reducing  the  computable  spatial  resolution  for 
a  given  window  size. 


Chapter  5 

Implementation 


The  framework  described  in  the  last  chapter  provides  the  mathematical  core  for  processing 
the  DSM  simulation.  Integration  of  the  DSM  simulation  into  the  IDMS  system  requires 
a  real-time  computer  architecture  to  meet  the  computational  constraints  of  an  interactive 
virtual  environment.  Sections  5.1  and  5.2  describe  the  hardware  and  sof  vare  architectures, 
respectively,  that  provide  the  computational  backbone  for  IDMS.  Section  5.3  explains  the 
flow  of  processing  from  the  initial  state  of  obtaining  model  data  files  to  the  state  of  real¬ 
time  interaction.  Section  5.4  proceeds  to  describe  the  real-time  system  loop.  Section  5.5 
explains  the  algorithms  and  data  structures  used  to  generate  an  interaction  force  both  on  the 
environment  and  the  user.  Lastly,  Section  5.6  describes  the  current  methods  of  generating 
data  models  for  simulation  in  IDMS. 

5.1  System  Architecture 

Several  high-performance  components  comprise  the  hardware  architecture  that  IDMS  runs 
upon.  The  choice  of  architecture  is  heavily  influenced  by  the  concurrent  run-time  model  used 
in  simulating  IDMS  (see  Section  5.2).  Figure  5-1  illustrates  the  IDMS  system  architecture 
and  the  flow  of  information  between  the  components.  Each  of  the  components  and  their 
connectivity  is  described  below. 

5.1.1  Human 

The  human  delivers  position  to  and  receives  force-feedback  from  the  haptic  Interface.  Visual 
feedba-'  3  delivered  to  the  human  by  the  visual  engine  through  a  color  display  and  the 
optional  use  of  StereoGraphics  CrystalEyes  shutter  glasses,  which  provide  a  stereo  display 
of  the  screen  to  the  user.  In  the  current  IDMS  virtual  environment  setting,  the  user  places 
herself  behind  the  haptic  interface  and  the  visual  display  is  positioned  directly  behind  the 
haptic  interface.  Mathematically  speaking,  the  origins  of  the  user,  haptic  interface,  and  visual 
display  reference  frames  are  placed  roughly  in  a  collinear  fashion  and  the  depth  axes  of  the 
three  frames  are  approximately  parallel.  The  visual  display  is  positioned  as  close  as  possible 
behind  the  haptic  interface  without  interfering  with  the  workspace  of  the  device.  Based  on 
empirical  trials  of  different  plans,  this  spatial  layout  allows  the  user  to  register  haptic  and 
visual  signals  in  a  perceptually  coherent  manner. 
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Figure  5-1:  System  architecture  for  IDMS.  Wide  arrows  represent  flow  of  sensorimotor  sig¬ 
nals;  thin  arrows  represent  flow  of  electronic  data. 

5.1.2  Haptic  interface 

The  haptic  interface  provides  information  about  the  Cartesian  position  of  the  endpoint  in  the 
form  of  encoder  data  for  each  of  the  haptic  interface  axes.  Motor  torques  applied  to  the  joints 
of  the  haptic  interface  are  mechanically  transmitted  through  the  device’s  structure,  resulting 
in  an  endpoint  force  applied  to  the  user.  The  Phantom  haptic  interface  is  u^ed  in  IDMS  to 
serve  these  purposes  and  is  described  below. 

The  Phantom  haptic  interface  [11,  12]  shown  in  Figure  5-2  is  a  three-degree-of-freedom 
force- reflecting  interface  that  relies  on  a  cable-driven  transmission  and  well-balanced  mech¬ 
anic'  for  high  fidelity  transmission  of  position  and  force  signals.  The  design  of  the  Phantom 
allows  users  to  execute  wrist-centered  motions  with  a  large  workspace.  Multiple  Phantoms 
can  be  interfaced  with  the  hand  allowing  multi-finger  interactions  with  the  virtual  environ¬ 
ment.  The  endpoint  of  the  Phantom  is  a  passive  three  degree-of-freedom  gimbal  which  allows 
for  torque-free  rotations.  This  feature  enables  the  Phantom  to  be  treated  as  a  point  force 
source  at  its  endpoint.  Various  tools,  passive  or  active,  can  be  inserted  into  the  gimbal  to 
model  different  types  of  interactions  with  the  world.  The  simplest  of  these  tools,  a  thimble, 
is  shown  more  clearly  in  the  Phantom  schematic  in  Figure  5-3  and  allows  the  user  to  apply 
and  receive  point  forces  at  the  fingertip.  In  the  context  of  surgical  simulation,  a  surgical  tool 
such  as  a  scalpel  or  tweezer  may  be  inserted  into  the  gimbal  to  reflect  the  desired  training 
situation.  For  example,  Figure  5-4  shows  an  instrumented  passive  tweezer  mounted  to  the 
gimbal.  Binary  contact  information  is  provided  through  an  electrical  switch  that  is  closed 
when  both  tweezer  sides  are  touching. 

5.1.3  Haptic  engine 

The  haptic  computational  engine  is  responsible  for  running  processes  that  simulate  the  dy¬ 
namics  of  the  virtual  environment  (environment  process),  servo  the  haptic  interface  (haptic 
process),  and  update  the  visual  process  on  the  visual  engine  with  the  current  environment 
state.  As  explained  in  Section  5.2,  the  haptic  and  environment  processes  are  run  as  one  com- 
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Figure  5-2:  The  Phantom  haptic  interface. 


bined  process  because  of  the  inherent  interactive  coupling  between  the  environment  dynamics 
and  the  haptic  interface.  A  120  MHz  Pentium-based  computer  running  Linux,  a  unix-based 
multitasking  operating  system,  is  used  to  serve  these  purposes. 


5.1.4  Visual  engine 

The  visual  engine  is  responsible  for  displaying  the  state  of  the  virtual  environment  at  a 
sufficient  refresh  rate  to  the  human  visual  system.  A  200  MHz  Silicon  Graphics  Indigo  II  Ex¬ 
treme  workstation,  used  as  the  IDMS  visual  engine,  is  designed  for  fast  graphics  computation 
because  its  architecture  has  an  integrated  graphics  pipeline  for  processing  low-level  graph¬ 
ics  routines.  The  visual  processes  for  IDMS  use  the  Silicon  Graphics  Openlnventor  library 
toolkit  built  upon  the  OpenGL  protocol  for  platform  independent  graphics  rendering.  All 
the  figures  in  this  text  which  display  visually  rendered  scenes  from  IDMS  depict  the  viewer 
window  that  is  standard  to  the  Openlnventor  library.  Incorporated  into  this  viewer  are  basic 
viewing  primitives  such  as  zooming,  panning,  rotating,  lighting,  and  shading  that  are  easily 
controllable  by  the  user.  Lastly,  data  is  communicated  between  the  haptic  and  visual  engines 
over  Ethernet  using  a  UDP  packet  level  connection,  ensuring  high  throughput  rates  of  data 
transfer. 
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Figure  5-3:  Schematic  of  Phantom  haptic  interface  (Courtesy  of  David  Brock,  MIT  Artificial 
Intelligence  Laboratory). 


5.2  Run-time  Model 

A  concurrent  [5,  p259]  model  is  utilized  as  the  run-time  software  architecture  for  IDMS  (see 
Figure  5-5).  Under  the  concurrent  model,  separate  operations  in  the  environment  run  in 
parallel  on  different  processors.  In  IDMS  two  main  processes,  which  handle  three  discrete 
subprocesses,  run  concurrently  on  the  haptic  and  visual  engines.  The  client  process,  running 
on  the  haptic  engine,  is  comprised  of  the  environment  and  haptic  processes  and  updates 
at  approximately  1  kHz.  The  environment  and  haptic  processes  are  lumped  together  as 
one  process  because  the  haptic  interaction  force  is  applied  to  the  haptic  interface  and  the 
environment  dynamics  concurrently  (see  Section  5.4).  The  server  process,  which  runs  on 
the  visual  engine  and  is  <.  ■'uivalent  to  the  visual  process,  receives  environment  data  from  the 
client  process  and  renders  the  visual  state  of  the  environment  at  approximately  30  Hz.  In 
order  to  decouple  loading  effects  between  the  client  and  server  processes,  a  separate  shared- 
memory  process  runs  on  both  the  haptic  and  visual  engines  to  handle  interprocessor  data 
communication.  The  client  process  updates  the  shared-memory  process  on  the  haptic  engine 
with  differential  changes  in  environment  data,  hence  minimizing  latency  in  the  time-critical 
client  loop.  The  two  individual  shared-memory  processes  run  synchronously  using  a  hand¬ 
shaking  protocol  and  serve  to  transfer  the  entire  visual  state  of  the  environment  at  30  Hz. 
Lastly,  the  server  process  runs  synchronously  with  the  shared-memory  process  on  the  visual 
engine  and  hence  also  updates  the  screen  at  30  Hz.  Because  Linux  is  not  a  preemptive 
operating  system,  the  shared-memory  process  on  the  haptic  engine  is  currently  initiated  by 
the  client  process  at  30  Hz.  The  use  of  a  preemptive  real-time  operating  system,  such  as 
LynxOS,  would  allow  each  process  to  be  time-sliced  during  each  unit  of  processing  time, 
allowing  for  more  transparent  decoupling  between  processes  (see  Section  7.2.1). 
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Figure  5-4:  Instrumented  tweezers  attached  to  Phantom  gimbal. 

A  concurrent  architecture  is  appealing  as  a  run-time  model  for  IDMS  for  the  following 
reasons. 

1.  Modal  dtcomposition 

By  separating  haptic  and  visual  software  onto  two  platforms,  issues  specific  to  each 
sensorimotor  modality  can  be  addressed  and  optimized  independently.  Assuming  mod¬ 
ular  programming  techniques  are  used,  changes  to  visual  rendering  software  will  not 
affect  the  performance  or  integrity  of  the  haptic  rendering  software.  Furthermore,  each 
platform  can  be  tailored  to  the  specific  hardware  needs  of  the  modality  it  represents. 
The  haptic  engine,  for  example,  requires  bus  slots  for  expansion  cards  to  accommod¬ 
ate  interfacing  with  strain  gauges,  amplifiers,  encoders,  and  other  analog  and  digital 
input/output.  Because  of  the  extensive  commercial  hardware  available  for  the  PC'  plat¬ 
form  and  the  memory-mapped  I/O  on  the  bus,  hardware  interfacing  is  greatly  facilit¬ 
ated.  Furthermore,  other  modalities,  such  as  audition,  can  be  addressed  independently 
of  currently  represented  modalities  without  necessarily  constraining  hardware  or  soft¬ 
ware. 

2.  Parallel  processing 

The  use  of  additional  processors  increases  computational  power  significantly  by  par¬ 
allelizing  computation.  Furthermore,  each  processor  can  be  matched  independently  to 
the  required  refresh  rate  of  the  modality  it  represents,  resulting  in  decoupling  'between 
rendering  rates  for  different  modalities.  As  stated  in  Section  2.2,  an  adequate  haptic 
display  requires  at  least  1  kHz  bandwidth  to  convincingly  transmit  tactile  and  force 
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HAPTIC  ENGINE  VISUAL  ENGINE 


Figure  5-5:  Concurrent  model  of  run-time  processing  for  IDMS. 


information.  Conversely,  a  visual  display  only  requires  at  minimum  a  throughput  rate 
of  8  to  10  Hz  and  a  latency  of  0.1  seconds.  Since  the  haptic  process  s  part  of  a  feedback 
control  system  that  includes  the  haptic  interface,  it  is  desirable  to  minimize  computa¬ 
tion  delay  in  order  to  maximize  force  display  bandwidth.  Computation  specific  to  the 
visual  modality  is  separated  onto  the  visual  engine,  thus  parallelizing  computation  of 
and  decoupling  loading  effects  between  the  haptic  and  visual  processes. 


5.3  Process  flow 

An  IDMS  session  follows  the  process  flow  diagram  shown  in  Figure  5-d.  The  database  of  pig 
and  fem  files  is  accessible  to  both  the  haptic  and  visual  engines1.  Thi  left  sequence  is  run 
by  the  haptic  engine;  the  sequence  on  the  right  is  executed  by  the  visual  engine.  When  both 
sequences  have  completed,  the  real-time  interactive  simulation  begins. 

The  client  process  parses  the  fem  file  and  constructs  a  complete  nodel  data  structure,  Ad, 
described  in  Section  3.3.  Once  the  model  is  created,  the  client  process  proceeds  to  set  the 
initial  conditions  of  the  model  in  the  environment.  Although  this  procedure  is  currently  em¬ 
bedded  within  the  simulation,  a  shell  could  be  written  that  allows  th  j  user  to  interactively  set 
the  initial  conditions  of  the  model.  The  haptic  interface  is  then  calibrated  and  geometrically 
registered  with  the  reference  frame  of  the  environment.  Finally,  tha  shared-memory  process 
on  the  haptic  engine  is  initiated  as  a  child  process  by  the  client  and  the  user  is  prompted  to 
begin  the  simulation. 

In  parallel,  the  server  (visual)  process  initiates  a  viewer  window  for  graphics  display  and 


'Each  model  is  comprised  of  a  pig  and  fe»  file  sharing  the  same  filename  prefix,  for  example  cube. pig 
and  cube.fes. 
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Figure  5-6:  Process  flow  of  an  IDMS  session.  The  sequence  on  the  left  is  executed  by  the 
haptic  engine;  the  sequence  on  the  right  is  executed  by  the  visual  engine.  The  interactive 
simulation  loop  begins  when  both  sequences  have  finished. 

forks  off  the  shared  memory  process  on  the  visual  .engine  as  a  child  process.  When  the  client 
process  reads  the  f  am  file,  the  server  is  notified  and  the  corresponding  pig  file  is  read  from 
the  database.  This  pig  file  is  used  to  create  an  object  data  structure  native  to  Open  Inventor 
for  storing  topology  and  geometry  information.  The  server  then  displays  the  object  in  the 
viewer  window.  The  user  proceeds  to  orient  the  object  in  the  viewer  window  so  that  it  is 
visually  registered  with  the  haptic  interface.  Graphical  editors  and  menus  allow  the  user 
to  easily  change  visual  properties  such  as  material  color,  display  mode  (shaded,  wireframe, 
or  vertices),  and  lighting  direction.  Stereo  viewing  is  a  built-in  option  to  the  OpenInven*or 
viewer  window  and  can  be  selected  dynamically2.  Once  the  visual  conditions  are  set,  the 
user  signals  the  client  process  to  begin  the  IDMS  simulation  loop.  Viewing  conditions  can  be 
changed  if  necessary  at  any  time  during  the  simulation. 

5.4  System  Loop 

Figure  5-7  depicts  in  detail  the  computation  blocks  that  are  executed  and  the  processes  they 
occupy  during  each  system  loop  of  IDMS.  The  DSM  simulation  blocks  are  each  represented 
by  a  thick  border  in  the  environment  process.  As  mentioned  in  Section  5.2,  the  environ¬ 
ment  and  haptics  processes  are  combined  as  one  process  because  both  are  synchronized  to 
the  interaction  force  computation  block  (Section  5.5.1).  Because  only  local  dynamics  are 

3  If  stereo  viewing  is  chosen,  the  user  can  interactively  choose  the  offset  value  between  the  left  and  right 
images  presented  to  the  shutter  glasses. 
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simulated,  the  environment  dynamics  and  haptic  display  can  be  updated  simultaneously  and 
still  meet  required  perceptual  refresh  rates  for  the  human  haptic  system.  As  referred  to  in 
Section  3.2.2,  however,  the  next  design  iteration  of  IDMS  might  involve  a  second  environment 
process  to  update  global  dynamics  at  a  slower  refresh  rate.  In  this  situation,  the  first  loop  in 
the  diagram  would  still  be  updated  quickly,  thus  maintaining  haptic  fidelity,  and  the  second 
loop  would  be  updated  less  often  for  atoms  and  elements  external  to  the  haptic  window. 


5.5  Interaction  Force  Calculation 

5.5.1  Master  Interaction  Object 

The  subsystem  of  IDMS  that  calculates  the  interaction  force,  the  master  interaction  object 
(MIO),  is  the  software  element  connecting  the  simulation  and  the  haptic  interface.  The 
MIO  models  in  software  the  contact  interaction  between  the  human  and  the  object  surface. 
The  MIO  inputs  position  information  from  the  haptic  interface  and  the  DSM  simulation  and 
outputs  a  force  signal  back  through  the  haptic  interface  and  the  DSM  simulation.  Each  haptic 
interface  that  is  connected  to  IDMS  is  assigned  a  different  MIO. 

In  system  engineering  terminology,  the  MIO  is  a  software  entity  representing  a  linear 
time-invariant  (LTI)  two-port  element.  There  are  energy  requirements  imposed  on  the  MIO 
by  the  virtual  environment.  TVs*.  ; elements  are  enumerated  below  and  refer  to  Figure 
5-8,  which  shows  the  power  flow  between  each  o?  the  two-port  elements  in  IDMS. 

1.  Causality 

In  order  to  preserve  integral  causality  on  each  of  the  atom  masses  in  the  DSM,  only 
forces  can  be  applied  as  inputs  to  them.  Given  that  the  haptic  interface  only  outputs 
position  information,  the  MIO  must  convert  position  input  into  force  output  to  satisfy 
the  causality  constraint  in  the  DSM  simulation.  Similarly,  the  haptic  interface  can  only 
apply  force  on  the  user.  Thus,  the  MIO  must  also  convert  position  output  from  the 
DSM  simulation  into  force  input  for  the  haptic  interface.  Similar  to  a  two- port  gyrator, 
the  MIO  transforms  an  effort  variable  input  from  port  A  into  a  flow  variable  output 
at  port  B;  furthermore,  a  flow  variable  input  at  port  B  is  transformed  into  an  effort 
variable  output  at  port  A.  Unlike  a  gyrator,  however,  the  MIO  uses  position  input  from 
both  the  first  port  (haptic  interface)  and  the  second  port  (DSM  simulation)  to  generate 
equal  and  opposite  force  output  on  both  ports.  Lastly,  the  MIO  can  store  energy  but  a 
gyrator  must  conserve  power  at  both  ports,  hence  it  is  unable  to  accumulate  energy. 

2.  Passivity 

As  mentioned  in  Section  2.2,  an  important  property  for  a  virtual  environment  is  passiv¬ 
ity  along  the  transmission  path  between  the  simulation  and  the  human.  Implementa¬ 
tion  of  the  MIO  using  software  models  of  passive  physical  elements  will  ensure  passive 
characteristics.  Early  experiments  with  potential  field  based  modeling  of  passive  envir¬ 
onments  approaches  utilized  a  non-passive  MIO,  resulting  in  the  undesirable  perceptual 
feel  of  an  active  environment. 

3.  Stiff  dynamics 

Depending  on  its  representation,  the  MIO  can  have  internal  dynamics.  In  the  context 
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Figure  5-7:  Breakdown  by  process  of  overall  system  loop.  The  environment  and  haptics 
processes  are  synchronized  to  run  at  the  same  speed  because  the  external  force  is  applied  to 
the  haptic  interface  and  the  simulation  dynamics  concurrently. 
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Figure  5-8:  Power  flow  between  two-port  elements  in  IDMS. 


of  deformable  media,  it  is  desirable  for  the  interaction  element  to  have  a  much  stiffer 
surface  impedance  than  that  present  in  the  model,  in  order  to  prevent  filtering  or 
attenuation  of  deformation  force  during  contact.  If  the  poles  of  the  interaction  element 
are  too  closely  located  to  those  of  the  model  being  simulated,  the  human  may  not  be 
able  to  distinguish  the  dynamics  of  the  interaction  element  from  the  dynamics  of  the 
simulated  media. 

Specifically,  the  MIO  is  defined  by  a  static  geometric  shape  that  is  dynamically  centered 
at  the  endpoint  of  the  haptic  interface.  This  shape  represents  the  geometry  of  the  real-world 
tool  being  modeled  for  interaction.  The  minimum  size  of  the  MIO  is  determined  by  the 
lowest  spatial  resolution  of  atoms  present  on  the  model  surface.  Too  small  a  size  for  the 
MIO  will  generate  a  nonsmooth  force  during  stroking.  Too  large  a  size,  however,  increases 
the  number  of  atoms  and  elements  that  need  to  be  simulated  to  generate  an  interaction  force, 
thus  increasing  computation  load.  These  tradeoffs  need  to  be  considered  carefully  when 
determining  the  spatial  resolution  and  MIO  size  for  a  particular  modeled  environment. 

During  contact,  atoms  that  are  positioned  internal  to  the  geometric  boundary  of  the  MIO 
receive  an  external  force.  A  reaction  force  is  applied  to  the  haptic  interface  equal  to  the 
negative  sum  of  external  forces  applied  on  DSM  atoms  by  the  MIO.  Figure  5-9  illustrates 
a  typical  interaction  sequence  using,  for  example,  a  spherical  MIO  shape.  Currently,  each 
atom  receives  a  force  proportional  to  its  internal  penetration  distance  normal  to  the  geometric 
surface,  simulating  an  elastic  contact.  Other  force  generating  functions  are  possible  as  long  as 
they  satisfy  the  energy  requirements  enumerated  above.  A  simple  collision  detection  algorithm 
is  used  to  find  those  atoms  of  the  model  that  are  internal  to  the  MIO  boundary.  Collision 
detection  with  the  MIO  is  computationally  efficient  because  only  point  intersections  need 
to  be  found.  Increasingly  complex  geometries  can  be  created  for  the  MIO  as  a  function  of 
processing  power  and  more  efficient  collision  detection  algorithms.  The  following  example 
illustrates  the  MIO  representation  of  fingertip  interaction. 

Example  1:  MIO  representation  of  fingertip 

Modeling  the  fingertip  as  a  haptic  interaction  element  is  accomplished  by  representing  the 
MIO  as  a  spherical  object.  A  sphere  is  an  ideal  shape  for  the  MIO  because  collision  detection 
and  force  generation  aie  fast  to  compute.  The  data  structure,  MIO,  shown  in  Table  5.1  is 
used  to  store  parameters  and  variables  for  a  spherical  MIO  of  radius  r  with  surface  stiffness 
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Figure  5-9:  Illustration  of  an  interaction  between  a  spherically-shaped  MIO  and  the  surface 
of  a  model. 

k. 


_ MIO _ 

p  absolute  position  of  haptic  interface  endpoint 
f  net  force  applied  on  environment 
k  boundary  stiffness 
r  radius  of  interaction  sphere 

H  radius  of  local  sphere 


Table  5.1:  Data  structure  for  MIO  representation  of  human  fingertip. 

The  following  algorithm  computes  the  collision  detection  and  calculation  of  interaction 
force  using  a  spherical  MIO.  This  algorithm  comprises  the  top  three  computation  blocks  of 
the  haptic  process  in  Figure  5-7.  A  look-up  table  of  square  root  values  is  used  to  reduce 
square  root  computation  time  by  a  factor  of  50%. 

=  0 

for  i=l:  NS 

d  =  ,4‘->p  -  MIO-* p 
r  =  |d| 

if  r  <  MlO-tr 

f  =  M10->k{M10->r  -  r)/(r)|fj 
.4'->f+=f 

=  f 

end 

end 

Note  that  currently  all  NS  atoms  are  scanned  for  possible  intersection  with  the  MIO.  The 
integration  of  three-dimensional  spatial  data  structures  (octrees)  into  IDMS  would  greatly 
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reduce  the  search  space  of  potential  collisions.  Off-line,  the  model  would  be  subdivided 
into  a  tree  of  octants  to  a  given  spatial  resolution.  A  preprocessing  step  would  assign  each 
surface  atom  to  a  leaf  octant  based  on  the  atom’s  equilibrium  position.  During  run-time,  the 
search  space  for  MIO  collision  is  consequently  reduced  to  leaf  octants  that  are  children  of 
the  smallest  octant  which  entirely  contains  the  MIO. 

5.5.2  Local  simulation 

As  stated  in  section  3.2.2,  the  feature  of  IDMS  enabling  real-time  simulation  is  the  ability 
to  simulate  the  region  of  DSM  media  local  to  the  interaction  point.  Local  simulation  is  an 
option  to  IDMS;  some  procedures  such  as  model  generation  utilize  the  DSM  algorithm  on 
all  model  atoms  and  elements  and  hence  require  the  local  simulation  option  to  be  turned  off. 
The  DSM  framework  allows  for  this  local  simulation  to  be  easily  implemented.  The  variable 
r/  in  the  MIO  data  structure  contains  the  radius  of  the  sphere,  centered  at  the  center  of  the 
MIO,  that  spatially  separates  atoms  and  elements  into  local  and  non-local  entities.  Atoms 
are  defined  as  local  if  their  absolute  position  lies  within  the  radius  of  the  local  sphere,  as 
illustrated  in  the  following  algorithm.  Note  that  internal  as  well  as  external  atoms  can  be 
defined  as  local;  hence  all  N  atoms  are  scanned. 

for  *  =  l  :  N 

d  =  ,4'-»p  -  MZO—tp 
r  =  |d| 

if  r  <  MIO-tri 

.A'-wpc  =  LOCAL 

end 

end 

If  local  simulation  is  on,  this  algorithm  is  executed  in  tandem  with  the  MIO  collision  detection 
algorithm  in  the  first  computation  block  of  the  haptic  process.  Atoms  are  first  tested  for 
inclusion  in  the  local  sphere  before  testing  for  collision  with  the  interaction  sphere  to  save 
computation.  Once  the  local  atoms  have  been  defined,  the  simulation  space  of  the  four  blocks 
of  the  DSM  algorithm  are  each  modified  as  shown  below. 

1.  Calculate  external  forces 

V4*  (0  <  i  <  N  -  I)  s.t.  ,4->spc  ==  local 

2.  Calculate  internal  element  forces 

V£'  (0  <  i  <  L  -  1)  s.t.  £->,40-wpc  ==  local  or  £-4Al-4spc  ==  LOCAL 

3.  Evaluate  time  derivatives  of  state  variables 

VA'  (0  <  i  <  N  -  1)  S.t.  ,4-45 pc  ==  LOCAL 

4.  Integrate  state  variables 

V4*  (Q  <  i  <  N  -  1)  s.t.  A — >spc  ==  local 

Note  in  block  two,  elements  are  indirectly  defined  as  local  if  either  one  of  their  bounding 
atoms  is  local.  As  mentioned  above,  the  use  of  spatial  data  structures  such  as  octrees  to  store 
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atom  location  would  greatly  reduce  time  spent  in  searching  through  the  list  of  model  atoms 
for  local  ones. 

5.6  Model  Generation 

5.6.1  Draping 

The  flexibility  of  IDMS  allows  simulation  of  any  model  that  adheres  to  the  fem  and  pig 
file  formats.  An  important  concern  to  be  addressed  is  the  creation  of  such  models  for  use  in 
IDMS.  Currently,  the  main  approach  to  model  generation  in  IDMS  is  based  on  draping  &  free¬ 
form  rectangular  viscoelastic  “skin”  slab  onto  a  parametrically  defined  rigid  form,  and  then 
saving  the  equilibrium  state  of  the  slab  as  the  newly  defined  model.  The  same  DSM  engine 
(Section  4.1)  is  used  to  simulate  the  physics  of  state  transition  from  the  initial  condition  to 
the  equilibrium  position.  Although  this  process  is  not  intended  to  be  real  time,  it  is  of  no 
consequence  because  the  haptic  display  channel  is  turned  off  during  the  draping  procedure. 

Draping  forces  are  generated  by  applying  attractive  forces  between  atoms  in  the  slab  and 
the  surface  of  the  parametric  object.  Thus,  the  skin  slab  wraps  itself  around  the  object  in  an 
energy-minimizing  fashion  similar  to  a  sheet  of  positive  charge  conforming  to  the  exterior  of 
a  negatively  charged  object.  Home  elements  are  added  to  each  of  the  NS  surface  atoms  on 
the  slab  once  the  state  is  saved,  in  effect  “hardening”  the  state  of  the  skin.  Finally,  the  new 
model  is  written  to  disk  as  pig  and  f  em  files  for  later  use  by  IDMS.  Although  the  resulting 
surface  is  not  closed  in  a  mathematical  sense,  the  surface  area  produced  is  sufficiently  large 
for  meaningful  interactions. 

The  three  stages  of  draping  are  described  in  detail  below  using  the  example  of  a  rect¬ 
angular  slab  and  a  spherical  form.  The  procedure  is  illustrated  in  the  six  frames  shown  in 
Figure  5-10. 

Stage  1:  Set  initial  conditions 

During  stage  one,  the  initial  conditions  of  the  slab  are  set  manually  so  that  it  is  centered 
above  the  center  of  the  rigid  form  and  exterior  to  the  surface  of  the  rigid  form. 

Stage  2:  Apply  attractive  forces  between  slab  and  form 

Once  the  slab  is  positioned  according  to  desired  initial  conditions,  the  DSM  engine  is  started 
and  the  attraction  forces  are  applied  between  the  slab  and  the  form,  illustrated  in  the  following 
algorithm.  The  spherical  form  has  a  radius  R /,  surface  stiffness  k'j,  attractive  stiffness  k°},  and 
a  viscosity  6/.  An  atom  external  to  the  form  surface  receives  an  attractive  force  proportional 
to  its  distance  away  from  the  surface  with  stiffness  kj.  The  material  parameters  k'j  and  6/ 
are  empirically  determined  to  produce  a  highly  inelastic  response  from  collision  between  slab 
atoms  and  the  form  surface,  consequently  minimizing  settling  time. 

Let  c  be  center  of  sphere 
for  t  =  1  :  N 

d  =  A'-^p  -  c 
r  =  |d| 
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Figure  5-10:  Example  depicting  three  stages  of  draping  usi'  a  rectangular  slab  and  a  spher¬ 
ical  form  (from  top  to  bottom,  left  to  right):  (1)  initi  state,  (2-5)  attraction,  and  (6) 
equilibrium. 
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if  r  >  Rj 

apply  attractive  forces 

^f+=_fco(r_iR/)d T 

else 

A‘-»f+=  k'j{Rf  -  -  hf(A'-*k\) 

apply  contact  forces 

end 

Stage  3:  Attach  home  elements  and  save  to  disk 

When  stage  three  is  reached,  the  slab  state  is  saved  and  home  elements  are  added  to  each 
surface  atom.  Finally,  the  new  spherically-shaped  model  is  written  to  disk  as  two  files,  the 
pig  file  and  the  f  em  file. 

for  i  =  1  :  N 

-4'->  Pe,  =  P 

if  A-WC  ==  EXTERNAL 

Add  new  home  atom  Ah' 

Ah'-*. Peq  =  A‘->  pe, 

Ah'~*  be  =  FIXED 

Add  new  home  element  Eh'  between  A1  and  Ak‘ 

end 

end 

Output  temp .  f  em  and  temp .  pig  files 
5.6.2  Other  methods 

Another  approach  to  model  generation  is  molding.  Using  the  haptic  display  channel,  the 
human  can  manually  create  shapes  by  manipulating,  squeezing,  depressing,  and  pinching 
existing  models.  Geometrically  simple  shapes  such  as  prisms  and  flat  sheets,  that  can  be  easily 
generated  algorithmically,  are  used  as  initial  models  to  mold.  This  approach  is  only  limited 
by  the  humans  creativity  in  model  generation.  Other  model  generation  techniques  are  also 
possible.  These  include  using  commercial  software  tools  such  as  AutoCAD  or  ProEngineer  for 
automatically  creating  surface  finite  element  meshes  for  arbitrary  topologies.  Furthermore, 
standalone  algorithms  that  mathematically  tessellate  the  surface  of  rigid  objects  with  the 
viscoelastic  tetrahedral  elements  should  be  feasible. 
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Chapter  6 

Results 


The  following  sections  illustrate  the  features,  performance,  and  versatility  of  IDMS  by  de¬ 
scribing  results  from  fundamental  interaction  tasks.  Visual  results  from  each  interaction  are 
presented  as  a  sequence  of  screen  images  from  the  IDMS  display1.  Force  and  position  histor¬ 
ies  of  the  MIO  object  are  also  provided  to  qualitatively  and  quantitatively  express  the  haptic 
display  performance. 


6.1  Basic  Interactions 

Each  of  the  basic  interactions  in  this  section  operate  on  the  homogeneous  DSM  media  frag¬ 
ment  shown  in  Figure  3-2.  The  haptic  display  is  refreshed  at  approximately  700  Hz  and  the 
visual  display  is  updated  at  30  Hz.  The  Cartesian  axes  of  the  fixed  IDMS  reference  frame 
are  shown  explicitly  in  the  first  frame  of  Figure  6-1.  All  position  measurements  are  taken 
with  respect  to  this  reference  frame.  The  element  parameters  are  k  =  15  and  b  =  0.2  and 
the  atom  mass  is  0.002.  The  top  surface  of  this  fragment  has  dimensions  of  4.00  inches  along 
x  and  3.46  inches  along  y.  The  surface  stiffness  of  the  MIO  was  empirically  found  to  be 
MXO-^k  =  400,  approximately  27  times  the  stiffness  of  the  elements  in  the  media.  Given 
the  relatively  high  stiffness  value  of  the  MIO,  the  deformation  force  is  transmitted  from  the 
simulation  to  the  user  with  negligible  filtering  of  information. 

During  initial  trial  sessions  with  IDMS,  a  transparent  sphere  was  used  to  visually  render 
the  MIO  object.  Interestingly,  this  action  was  visually  disconcerting  because  part  of  the 
sphere  would  disappear  beneath  the  top  surface  of  the  mesh  due  to  atoms  penetrating  the 
MIO  surface.  Users  tend  to  focus  on  the  media  deformation  itself  rather  than  the  MIO  object 
while  in  contact.  Consequently,  a  small  diamond  shaped  object  was  used  instead  because  it 
provided  an  unobstructed  view  of  the  deformation  and  removed  the  disturbance  due  to  visual 
interference  between  the  MIO  and  the  media.  Taking  into  account  the  DSM  element  length 
of  1  inch,  the  radius  of  the  MIO  was  empirically  optimized  to  be  MIO-+r  =  1.5  inches  for 
modeling  the  human  fingertip.  Although  this  radius  is  approximately  three  times  the  radius  of 
a  fingertip,  users  do  not  perceive  the  actual  MIO  size  due  to  the  small  diamond-shaped  visual 


‘In  the  following  figures  where  appropriate,  the  viewer  display  is  shown  in  reverse  video  and  all  objects 
are  shown  in  wireframe  to  clarify  deformation  results 
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representation.  The  absence  of  collocation  between  the  haptic  and  visually  displays  and  the 
poor  kinesthetic  discriminability  of  the  human  [5,  pl70]  allow  the  user  to  successfully  register 
haptic  and  visual  cues  using  primarily  haptic  force  and  visual  deformation  information. 

6.1.1  Palpation 

A  typical  finger  palpation  sequence  is  depicted  in  Figure  6-1.  The  force  and  position  histories 
for  the  MIO  are  shown  in  Figure  6-2.  As  expected,  fx  and  fy  are  negligible  relative  to 
deformation  force  normal  to  the  surface,  fz.  The  peak  value  for  fz  reached  is  approximately 
1.7  N.  Note  that  fz  becomes  nonzero  at  approximately  z  =  1.5,  which  is  the  radius  of  the 
MIO. 

The  force  versus  displacement  relationship  applied  to  the  haptic  interface  for  a  palpation 
interaction  is  shown  in  Figure  6-3.  The  measurement  data  is  fitted  in  a  linear  least  squares 
sense  to  illustrate  the  stiffness  behavior  of  the  media.  The  stiffness  increases  initially  as 
more  atoms  come  into  contact  with  the  MIO  and  then  transitions  into  a  linear  function 
as  the  number  of  intersection  atoms  levels  off.  The  curve  illustrates  that  the  contribution 
of  deformation  force  by  home  elements  of  intersection  atoms  is  dominant  relative  to  the 
force  provided  by  non-intersection  atoms.  The  use  of  a  haptic  window  exploits  this  rapidly 
diminishing  contribution  of  force  by  conserving  processing  power  for  local  atoms. 

6.1.2  Stroking 

In  contrast  to  palpation,  stroking  involves  motion  of  the  MIO  lateral  to  the  surface  while 
is  in  contact  with  the  media.  A  sample  stroking  sequence  is  shown  in  Figure  6-4.  At  a. 
approximately  fixed  position  of  y  =  1.5  inches,  the  MIO  object  is  brought  into  contact  with 
the  media  surface,  laterally  moved  at  constant  z  in  the  direction  of  increasing  x,  and  then 
brought  out  of  contact  with  the  media.  In  order  to  illustrate  the  effect  of  speed  on  lateral 
force  interaction,  two  cases  differing  in  lateral  surface  speed  are  discussed  below. 

Slow  lateral  surface  speed 

As  shown  in  Figure  6-5,  between  1  w  1  sec  and  (  as  3  sec,  the  MIO  object  lowers  and 
deforms  the  media  surface,  increasing  the  normal  force  to  a  value  of  fz  ss  1.3  N.  As  the  MIO 
accelerates  in  the  positive  x  direction  across  the  media,  the  opposing  lateral  force  fx  increases 
to  -0.1  N  due  to  the  inertia  of  atoms  along  the  MIO  path  that  needs  to  be  overcome.  As  the 
MIO  decelerates  at  x  s»  3  inches,  the  lateral  force  changes  sign  to  reflect  the  buildup  of  inertia 
behind  the  MIO  object. 

Fast  lateral  surface  speed 

Figure  6-6  displays  the  MIO  force  and  position  histories  for  the  same  stroking  sequence  shown 
in  Figure  6-4,  with  the  exception  that  the  surface  is  traversed  three  times  while  the  MIO  is 
in  contact.  Although  the  average  normal  force  fz  is  similar  to  that  of  the  previous  case,  the 
maximum  lateral  force  fx  reaches  a  maximum  magnitude  of  almost  2  N  during  each  traversal 
due  to  the  increased  lateral  speed.  Similarly,  the  lateral  force  perpendicular  to  the  traversal 
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MIO  z  force  vs.  z  position  (measured) 


MIO  z  force  vs.  z  position  (third  order  linear  least-squares  fit) 


Figure  6-3:  Interaction  stiffness  normal  to  surface  for  palpation:  (top)  measured,  (bottom) 
linear  least  squares  fit. 
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path,  /y,  also  increases  dramatically  because  of  the  rapid  transfer  of  momentum  to  atoms  as 
they  are  pushed  aside. 

6.1.3  Plucking 

Using  the  tweezers  shown  in  Figure  5-4  as  an  interaction  tool,  the  surface  can  be  plucked, 
manipulated,  and  held.  The  MIO  is  slightly  modified  to  model  the  grasping  ability  of  the 
tweezers  as  follows.  If  the  MIO  is  in  contact  with  the  surface  when  the  tweezers  are  closed, 
a  sorting  algorithm  is  executed  to  find  the  nearest  atom,  of  those  that  are  in  contact  with 
the  MIO,  to  the  center  of  the  MIO.  A  new  element  £  is  created  that  connects  the  nearest 
atom  and  the  center  of  the  MIO  with  an  equilibrium  length  equal  to  the  distance  between  the 
atom  and  the  MIO  center  at  the  time  of  closure.  The  stiffness  of  this  element  is  assigned  the 
same  stiffness  as  that  of  the  MIO  surface  to  simulate  a  rigid  attachment  with  the  atom.  If 
the  tweezers  are  opened,  the  connecting  element  is  deleted,  hence  allowing  the  contact  atom 
to  return  to  equilibrium.  A  sample  plucking  sequence  is  shown  in  Figure  6-7. 

Figure  6-8  shows  the  position  and  force  histories  of  the  MIO  during  an  example  plucking 
sequence  with  a  media  fragment.  While  the  MIO  is  in  contact  with  the  media,  the  user 
closes  the  tweezers  hence  creating  the  connecting  element  between  the  nearest  contact  atom 
and  the  MIO  center.  When  the  MIO  is  no  longer  in  contact  with  the  surface,  the  MIO  force 
represents  the  connecting  element  force.  At  t  «  4  sec,  the  tweezers  are  opened  and  the  MIO 
force  instantly  falls  to  zero  magnitude.  There  is  a  slight  perturbation  in  MIO  position  as  the 
human  stabilizes  after  receiving  a  step  function  in  force. 

6.2  Local  Simulation 

The  sequence  shown  in  Figure  6-9  illustrates  the  stroking  interaction  with  a  higher  resolution 
media  fragment.  The  curvature  of  the  fragment  was  generated  by  draping  it  onto  a  spherical 
form  and  consequently  saving  the  state  of  the  new  model.  The  local  simulation  option  is 
enabled  using  a  haptic  window  radius  of  MlO—tri  =  2.25  inches.  Although  the  haptic  display 
refresh  rate  suffers  slightly  in  this  particular  example  due  to  the  increase  in  spatial  density 
of  atoms,  the  refresh  rate  is  largely  uncoupled  from  the  size  of  the  media  fragment.  Recall 
from  Section  5.5.2,  some  coupling  is  still  present  because  the  collision  detection  algorithm 
searches  the  entire  atom  list  for  potential  collision  atoms.  As  described  previously,  however, 
complete  uncoupling  would  occur  if  octrees  were  used  to  spatially  segment  the  DSM  model. 


6.2.  LOCAL  SIMULATION 


Figure  6-6:  Position  and  force  histories  for  the  MIO  during  a  stroking  interaction  using 
fast  lateral  surface  speed. 


JJ 


ttN  ■ 

• 

( ^  - 

3 , 

1*1 

it 

LV 

<* 

V' 

- 

■'  t 

'■J 

t 

• 

■  *, 

1 

i 

j 

A 

....  * 

*  * 

_ '•  V 

> 

•» 

« 

I  r 

'  * 

Li 

* 

•• 

,  * 

% 

V 

i  ] 

- 

f  sl 

u 

* 

<  i 

i 

0 

: 

4 

[uence 

ght). 


Chapter  7 

Conclusions 


7.1  Summary 

In  summary,  Interactive  Deformable  b.edia  Simulator  (IDMS)  is  a  real-time  dynamic  simu¬ 
lation  that  allows  users  to  interact  both  haptically  and  visually  with  deformable  media  in  a 
virtual  environment  setting.  Haptic  feedback  is  provided  through  the  Phantom  haptic  inter¬ 
face  and  visual  feedback  is  presented  by  Openlnventor  based  graphics  software  on  the  Silicon 
Graphics  Indigo  II  Extreme  workstation.  A  finite  element  approach  is  utilized  to  model  the 
structural  properties  of  deformable  media.  Real-time  computation  is  achieved  by  (a)  using 
a  surface  representation  coupled  with  a  surface  restoring  force  to  encapsulate  volumetric 
properties:  (b)  conserving  computation  power  by  rendering  media  local  to  the  interaction 
point:  and  (c)  evaluating  the  simulation  within  the  topology  of  the  model.  The  discretized 
representation  of  the  media  facilitates  creation,  modification,  and  storage  of  model  data.  The 
Master  Interaction  Object  (MIO)  is  a  fixed  geometric  shape  that  efficiently  calculates  inter- 
ac  on  force  based  on  point  intersection  and  surface  penetration  distance  normal  to  the  MIO 
boundary.  Other  interaction  modalities  such  as  tweezers  are  easily  integrated  into  IDMS 
by  appropriate  modification  of  the  MIO  force  generation  algorithm.  The  following  section 
describes  further  research  that  would  enhance  the  performance  and  functionality  of  IDMS. 

7.2  Future  Work 

7.2.1  Computational  architecture 

The  DSM  framework  in  IDMS  is  highly  amenable  to  a  parallel  processing  architecture.  Ad¬ 
ditional  processors  could  be  assigned  mutually  exclusive  sections  of  media  for  simulation 
purposes.  Model  data  would  be  stored  in  common  memory  to  ensu  *ata  coherency  and 
to  facilitate  communication  of  data  between  processors.  Furthermore  al-time  preempt¬ 
ive  operating  system  such  as  LynxOS  for  the  haptic  engine  is  critical  to  evenly  distribute 
processing  between  the  haptic  and  environment  processes  and  the  shared  memory  process. 
Currently,  under  Linux,  the  haptic  process  signals  the  shared  memory  process  to  begin  a  data 
transfer  to  the  visual  engine  at  30  Hz.  Although  not  a  factor  for  simple  models,  the  delay 
induced  by  transferring  visual  information  for  complex  models  may  become  haptically  no¬ 
ticeable  at  the  30  Hz  frequency.  A  real-time  operating  system  will  allow  for  context  switching 
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between  processes  at  much  higher  service  rates  than  possible  in  Linux,  removing  the  need 
for  synchronization  between  haptic  and  shared  memory  processes. 


7.2.2  Dynamics  Simulation 

In  order  to  achieve  real-time  response,  local  simulation  is  utilized  by  IDMS  to  calculate  in¬ 
teraction  force.  For  the  class  of  media  addressed  in  this  thesis,  this  technique  is  sufficient  for 
creating  realistic  interactions.  Media  in  which  local  interactions  are  more  globally  propagated 
throughout  the  media  may  require  additional  concurrent  dynamics  processes  to  update  the 
model  state.  These  “global  dynamics”  processes  may  run  at  slower  update  rates  than  the 
“local  dynamics”  process  depending  on  the  frequency  content  present  in  the  media.  This  syn¬ 
chronous  execution  of  multiple  time-scale  dynamics  processes  fits  easily  into  the  concurrent 
run-time  architecture  of  IDMS. 


7.2.3  Spatial  data  structures 

Integrating  spatial  data  structures  into  IDMS  will  greatly  improve  computation  performance. 
Currently,  the  entire  list  of  surface  atoms  is  traversed  to  check  for  collision  detection  with  the 
MIO.  A  more  efficient  approach  would  be  to  segment  the  model  into  a  multi-resolution  spatial 
hierarchy.  Octrees  and  bounding  boxes  would  eas’ly  serve  these  purposes.  This  procedure 
could  be  done  as  a  preprocessing  step  to  IDMS  or  could  be  incorporated  into  the  pig  and 
f  em  data  file  formats. 


7.2.4  Haptic  interactions 

In  addition  to  the  fundamental  haptic  interactions  of  palpation,  stroking,  and  plucking,  other 
more  complex  haptic  interactions  should  be  modeled  and  implemented.  Each  interaction  rnay 
involve  independent  modification  of  the  interaction  force  algorithm  or  require  a  special  tool 
to  be  inserted  in  the  Phantom  gimbal  in  order  to  model  the  real  world  interaction.  Examples 
of  potential  interactions  include  sewing,  suturing,  piercing,  and  incising.  Surface  effects  such 
a.s  friction  and  stiction  could  also  be  implemented  as  external  forces  applied  to  individual 
atoms  and  the  MIO. 


7.2.5  Model  generation  and  filling 

In  order  to  exploit  the  variety  of  models  that  IDMS  can  simulate,  more  advanced  techniques 
need  to  be  created  for  generating  model  geometries  and  assigning  values  to  physical  para¬ 
meters  for  atoms  and  elements.  As  described  in  Section  3.2.1,  a  haptic  interface  with  an 
instrumented  end  effector  could  be  used  to  measure  surface  impedance  and  geometry  data. 
The  home  elements  provide  a  compact  representation  of  surface  impedance  and  hence  meth¬ 
ods  for  assigning  home  element  values  based  on  measurement  data  need  to  be  developed. 
Depending  on  the  context,  these  scanning  and  filling  techniques  may  be  required  to  be  non¬ 
destructive  to  preserve  the  state  of  the  real  object. 


7.2.  FUTURE  WORK 
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7.2.6  Visualization 

Currently,  the  media  is  visually  displayed  in  either  shaded  or  wireframe  graphics  modes. 
In  the  shaded  mode,  each  polygon  specified  by  the  pig  file  is  uniformly  shaded  based  on 
lighting  direction,  In  wireframe  representation,  only  each  surface  element  is  displayed.  In 
each  of  these  modes,  the  surface  smoothness  suffers  for  large  deformations  due  to  the  spatial 
discretization  of  atoms.  Two-dimensional  paramet'-.c  mU,  potation  polynomials  could  be 
incorporated  utilizing  the  positions  of  the  surface  atom."  .  control  points  to  provide  a  visually 
smooth  deformation.  B-splines  are  especially  suited  for  loc‘ !  deforr.  -Cion  because  they  utilize 
a  set  of  blending  functions  that  locally  influence  shape  changes,  depending  only  on  neighboring 
control  points  to  produce  changes  in  curvature  [14]. 

Depending  on  the  object  being  modeled,  texture  mapping  may  add  significant  realism 
to  the  visual  display.  Skin,  for  example,  could  be  texture  mapped  onto  biological  objects 
to  enhance  visual  feeuback.  The  SGI  Indigo  II  Extreme  platform  cannot  handle  real-time 
texture  mapping  and  thus  more  powerful  computation  hardware  would  be  necessary.  The 
SGI  Reality  Engine,  a  multiprocessor  based  hardware  architecture,  would  ideally  serve  these 
and  other  graphics  intensive  purposes. 
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Notation 

§ 


Symbol  Meaning  (units  if  applicable) 


IDMS 

DSM 

MIO 

P 

Peg 

Xl 

X2 

f 

num 

be 

sc 

spe 

m 

k 

b 

l 

leq 

r 

ri 

Rf 

kf 

kf 

bf 

T 

A 

E 

M 

MIO 


N 


Interactive  Deformable  Media  Simulator 

Discrete  Simulation  Model 

Master  Interaction  Object 

absolute  position 

absolute  equilibrium  position 

differential  position 

differential  velocity 

force 

number  in  sequence 

boundary  condition  (fixed  or  free) 

surface  condition  (external  or  internal) 

spatial  condition  (local  or  non-local) 

mass 

stiffness 

viscosity 

length 

equilibrium  length 
radius 
local  radius 
spherical  form  radius 
form  surface  stiffness 
form  attractive  stiffness 
form  surface  viscosity 
simulation  step  size  (sec) 
atom  data  structure 
element  data  structure 
model  data  structure 
MIO  data  structure 

dereferencing  operator,  A—*m  is  the  mass  stored  in  atom  A 
number  of  model  atoms 
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number  of  model  elements 
number  of  model  surface  atoms 


Appendix  B 

Data  File  Formats 


B.l  pig  files 

B.l.l  Description 

The  pig  file  description  is  one  of  several  widely  used  data  file  formats  for  describing  three- 
dimensional  polyhedra.  These  file  formats  are  used  for  a  variety  of  purposes  including 
computer-aided  design,  computer  graphics,  and  modeling.  A  brief  description  of  the  pig 
file  format  is  provided  here.  The  pig  file  format  is  essentially  a  subset  of  the  substantially 
more  versatile  obj  file  format  provided  by  Wavefront  Technologies  [25],  that  additionally 
allows  for  free-form  curves  and  surfaces,  materia!  properties,  nested  grouping  of  objects,  and 
many  other  graphics  attributes. 

The  format  of  a  pig  file,  shown  in  Figure  B-l  is  a  listing  of  vertices  and  polygons  that 
comprise  the  polyhedra,  each  listed  on  a  separate  line.  The  first  line  of  the  file  contains,  in 
order,  the  ASCII  name  of  the  object,  the  number  of  vertices,  and  the  number  of  polygons. 
Each  vertex  entry  has  the  x,  y,  and  z  coordinates  that  define  its  position.  Each  polygon  entry 
contains  the  number  of  sides  and  a  list  of  vertex  numbers  that  comprise  the  polygon,  the  first 
vertex  being  numbered  0.  A  whitespace  separates  the  first  line,  the  listing  of  vertices,  and 
the  listing  of  polygons. 

{name}  {nv}  {np} 

{*}  {y>  {*} 

M  {y}  {*} 

{number  of  sides}  {vl}  {v2}  ...{vn} 

{number of  sides}  {vl}  {v2}  ...{vn} 


Figure  B-l:  pig  file  format. 
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B.1.2  Example:  hex_4x5_home.plg 

The  pig  file  for  the  fragment  of  media  shown  in  Figure  3-1  is  shown  below, 
hex  22  28 

0.500  0.000  0.000 

1.500  0.000  0.000 

2.500  0.000  0.000 

3.500  0.000  0.000 
0.000  0.866  0.000 
1.000  0.866  0.000 
2.000  0.866  0.000 
3.000  0.866  0.000 
4.000  0.866  0.000 
0.500  1.732  0.000 

1.500  1.732  0.000 

2.500  1.732  0.000 

3.500  1.732  0.000 
O.COO  2.598  0.000 
1.000  2.598  0.000 
2.000  2.598  0.000 
3.000  2.598  0.000 
4.000  2.598  0.000 
0.500  3.464  0.000 

1.500  3.464  0.000 

2.500  3.464  0.000 

3.500  3.464  0.000 

3  0  15 
3  12  6 
3  2  3  7 
3  4  5  9 
3  5  4  0 
3  5  6  10 
3  6  5  1 
3  6  7  11 
3  7  6  2 
3  7  8  12 
3  8  7  3 
3  9  10  14 
3  10  9  5 
3  10  11  15 
3  11  10  6 
3  11  12  16 
3  12  11  7 


BA.  PIG  FILES 
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3  13  14  18 
3  14  13  9 
3  14  15  19 
3  15  14  10 
3  15  16  20 
3  16  15  11 
3  16  17  21 
3  17  16  12 
3  18  14  19 
3  19  15  20 
3  20  16  21 


€ 


i 
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B.2  fem  files 

B.2.1  Description 

The  fem  file  format  was  developed  for  IDMS  to  describe  networks  of  atoms  and  connecting 
constitutive  elements.  This  format  changes  as  increases  in  modeling  complexity  within  IDMS 
require  more  physical  parameters  or  initial  conditions  to  be  specified. 

The  specific  format  of  the  fem  file  is  shown  in  Figure  B-2.  The  first  line  of  the  file  contains, 
in  order,  the  name  of  the  object,  the  number  of  atoms,  and  the  number  of  elements.  Each 
atom  entry  has  the  x,  y,  and  z  coordinates  that  define  the  atom’s  initial  position;  the  atom 
mass;  the  boundary  condition  (fixed  or  free);  and  the  surface  condition  (on  surface  or  not). 
Each  element  entry  contains  the  numbers  of  the  two  atoms  the  element  connects,  the  element 
stiffness,  and  the  element  viscosity.  The  first  atom  in  the  atom  list  is  numbered  zero  A 
whitespace  separates  the  first  line,  the  listing  of  atoms,  and  the  listing  of  elements. 

{name}  {na}  {ne} 

{x}  {y}  {z}  {mass}  {boundary  condition}  {surface  condition} 

{x}  {y}  {z}  {mass}  {boundary  condition}  {surface  condition} 

{al}  {a2}  {stiffness}  {viscosity} 

{al}  {a2}  {stiffness}  {viscosity} 


Figure  B-2:  fem  file  format. 


B.2. 2  Example:  hex.4x5Jiome.fem 

The  fem  file  for  the  fragment  of  media  shown  in  Figure  3-1  is  shown  below.  Note  that  home 
atoms  and  home  elements  are  included. 


hex  58  141 


0.500  0.000  0.000 

1.500  0.000  0.000 

2.500  0.000  0.000 

3.500  0.000  0.000 
0.000  0.866  0.000 
1.000  0.866  0.000 
2.000  0.866  0.000 
3.000  0.866  0.000 
4.000  0.866  0.000 
0.500  1.732  0.000 


0.002000 
0.002000 
0.002000 
0.002000 
0.002000 
0 . 002000 
0.002000 
0.002000 
0.002000 
0 . 002000 


1001 

1001 

1001 

1001 

1001 

1001 

1001 

1001 

1001 

1001 


2001 

2001 

2001 

2001 

2001 

2001 

2001 

2001 

2001 

2001 
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1.500 

1.732 

0.000 

0 . 002000 

1001 

2001 

2.500 

1.732 

0.000 

0 . 002000 

1001 

2001 

3.500 

1.732 

0.000 

0 . 002000 

1001 

2001 

0.000 

2.598 

0.000 

0 . 002000 

1001 

2001 

1.000 

2.598 

0.000 

0.002000 

1001 

2001 

2.000 

2.598 

0.000 

0.002000 

1001 

2001 

3.000 

2.598 

0.000 

0 . 002000 

1001 

2001 

4.000 

2.598 

0.000 

0 . 002000 

1001 

2001 

0.500 

3.464 

0.000 

0 . 002000 

1001 

2001 

1.500 

3.464 

0.000 

0 . 002000 

1001 

2001 

2.500 

3.464 

0.000 

0 . 002000 

1001 

2001 

3.500 

3.464 

0.000 

0 . 002000 

1001 

2001 

0.500 

0.000 

0.000 

0.002000 

1000 

2000 

1.500 

0.000 

0.000 

0 . 002000 

1000 

2000 

2.500 

0.000 

0.000 

0 . 002000 

1000 

2000 

3.500 

0.000 

0.000 

0 . 002000 

1000 

2000 

0.000 

0.866 

0.000 

0 . 002000 

1000 

2000 

1.000 

0.866 

0.000 

0 . 002000 

1000 

2000 

2.000 

0.866 

0.000 

0 . 002000 

1000 

2000 

3.000 

0.866 

0.000 

0 . 002000 

1000 

2000 

4.000 

0.866 

0.000 

0 . 002000 

1000 

2000 

0.500 

1.732 

0.000 

0 . 002000 

1000 

2000 

1.500 

1.732 

0.000 

0 . 002000 

1000 

2000 

2.500 

1.732 

0.000 

0.002000 

1000 

2000 

3.500 

1.732 

0.000 

0 . 002000 

1000 

2000 

0.000 

2.598 

0.000 

0 . 002000 

1000 

2000 

1.000 

2.598 

0.000 

0 . 002000 

1000 

2000 

2.000 

2  598 

0.000 

0 . 002000 

1000 

2000 

3.000 

2.598 

0.000 

0.002000 

1000 

2000 

4.000 

2.598 

0.000 

0 . 002000 

1000 

2000 

0.500 

3.464 

0.000 

0 . 002000 

1000 

2000 

1.500 

3.464 

0.000 

0 . 002000 

1000 

2000 

2.500 

3.464 

0.000 

0 . 002000 

1000 

2000 

3.500 

3.464 

0.000 

0 . 002000 

1000 

2000 

0.500  0.289  -0.817  0.002000  1001  2000 

1.500  0.289  -0.817  0.002000  1001  2000 

2.500  0.289  -0.817  0.002000  1001  2000 
0.000  1.155  -0.817  0.002000  1001  2000 
1.000  1.155  -0.817  0.002000  1001  2000 
2.000  1.155  -0.817  0.002000  1001  2000 
3.000  1.155  -0.817  0.002000  1001  2000 
0.500  2.021  -0.817  0.002000  1001  2000 

1.500  2.021  -0.817  0.002000  1001  2000 

2.500  2.021  -0.817  0.002000  1001  2000 
0.000  2.887  -0.817  0.002000  1001  2000 
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1.000  2.887  -0.817  0.002000  1001  2000 
2.000  2.887  -0.817  0.002000  1001  2000 
3.000  2.887  -0.817  0.002000  1001  2000 


<S) 

(§> 

v+j 


0  1  15.000000  0.200000 

1  2  15.000000  0.200000 

2  3  15.000000  0.200000 

4  5  15.000000  0.200000 

5  6  15.000000  0.200000 

6  7  15.000000  0.200000 

7  8  15.000000  0.200000 

9  10  15.000000  0.200000 

10  11  15.000000  0.200000 

11  12  15.000000  0.200000 

13  14  15.000000  0.200000 

14  15  15.000000  0.200000 

15  16  15.000000  0.200000 

16  17  15.000000  0.200000 

18  19  15.000000  0.200000 

19  20  15.000000  0.200000 

20  21  15.000000  0.20000C 
0  4  15.000000  0.200000 

0  5  15.000000  0.200000 

4  9  15.000000  0.200000 

5  9  15.000000  0.200000 
1  5  15.000000  0.20000C 

1  6  15.000000  0.200000 

5  10  15.000000  0.200000 

6  10  15.000000  0.200000 

2  6  15.000000  0.200000 

2  7  15.000000  0.200000 

6  11  15.000000  0.200000 

7  11  15.000000  0.200000 

3  7  15.000000  0.200000 
3  8  15.000000  0.200000 

7  12  15.000000  0.200000 

8  12  15.000000  0.200000 

9  13  15.000000  0.200000 

9  14  15.000000  0.200000 

13  18  15.000000  0.200000 

14  18  15.000000  0.200000 

10  14  15.000000  0.200000 
10  15  15.000000  0.200000 

14  19  15.000000  0.200000 

15  19  15.000000  0.200000 


» 
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11  15  15.000000  0.200000 

11  16  15.000000  0.200000 

15  20  15.000000  0.200000 

16  20  15.000000  0.200000 

12  16  15.000000  0.200000 
12  17  15.000000  0.200000 

16  21  15.000000  0.200000 

17  21  15.000000  0.200000 
0  22  15.000000  0.200000 

1  23  15.000000  0  200000 

2  24  15.000000  0.200000 

3  25  15.000000  0.200000 

4  26  15.000000  0.200000 

5  27  15.000000  0.200000 

6  28  15.000000  0.200000 

7  29  15.000000  0.200u00 

8  30  15.000000  0.200000 

9  31  15.000000  0.200000 

10  32  15.000000  0  200000 

11  33  15.000000  0  200000 

12  34  15.000000  0  200000 

13  35  15  OOOOOO  0  200000 

14  36  IS  OOOOOO  0  200000 

15  37  15  OOOOOO  0  200000 

4  £  OQ  «  C  AAAAV*  ^ 

40  JO  i  J  •  VWWV  U  4WWV 

17  39  15  000000  0  200000 

18  40  15.000000  0  200000 

19  41  15.000000  0  200000 

20  42  15.000000  0  200000 

21  43  15.000000  0.200000 
44  0  15.000000  0.200000 
44  1  15.000000  0.200000 

44  5  15.000000  0  200000 

45  1  15.000000  0.200000 
45  2  15.000000  0.200000 

45  6  15.000000  0.200000 

46  2  lb. 000000  0.200000 
46  3  15.000000  0.200000 

46  7  15.000000  0.200000 

47  4  15.000000  0.200000 
47  5  15.000000  0.200000 

47  9  15.000000  0.200000 

48  5  15.000000  0.200000 
48  6  15.000000  0.200000 
48  10  15.000000  0.200000 


» 


» 


» 


i 


» 


A 
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49  6  15.000000  0.200000 
49  7  15 . 000000  0 . 200000 

49  11  15.000000  0.200000 

50  7  15.000000  0.200000 
50  8  15.000000  0.200000 

50  12  15.000000  0.200000 

51  9  15.000000  0.200000 
51  10  15.000000  0.200000 

51  14  15.000000  0.200000 

52  10  15.000000  0.200000 
52  11  15.000000  0.200000 

52  15  15.000000  0.200000 
5a  11  15.000000  0.200000 

53  12  15.000000  0.200000 

53  16  15.000000  0.200000 

54  13  15.000000  0.200000 
54  14  15.000000  0.200000 

54  18  15.000000  0.200000 

55  14  15.000000  0.200000 
55  15  15.000000  0.200000 

55  19  15.000000  0.200000 

56  15  15.000000  0.200000 
56  16  15.000000  0.200000 

56  20  15.000000  0.200000 

57  16  15.000000  0.200000 
57  17  15.000000  0.200000 
57  21  15.000000  0.200000 

44  45  15.000000  0.200000 

45  46  15.000000  0.200000 

47  48  15.000000  0.200000 

48  49  15.000000  0.200000 

49  50  15.000000  0.200000 

51  52  15.000000  0.200000 

52  53  15.000000  0.200000 

54  55  15.000000  0.200000 

55  56  15.000000  0.200000 

56  57  15.000000  0.200000 
44  47  15.000000  0.200000 

44  48  15.000000  0.200000 

47  51  15.000000  0.200000 

48  51  15 .000000  0.200000 

45  48  15.000000  0.200000 
45  49  15.000000  0.200000 

48  52  15.000000  0.200000 

49  52  15.000000  0.200000 
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46  49  15.000000  0.200000 
46  50  15.000000  0.200000 

49  53  15.000000  0.200000 

50  53  15.000000  0.200000 

51  54  15.000000  0.200000 

51  55  15.000000  0.200000 

52  55  15.000000  0.200000 

52  56  15.000000  0.200000 

53  56  15.000000  0.200000 
53  57  15.000000  0.200000 


I '  * 
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